Re: CVS commit: src/sys/uvm

2010-11-24 Thread YAMAMOTO Takashi
hi,

> On Thu, Nov 25, 2010 at 04:18:25AM +0000, YAMAMOTO Takashi wrote:
>> hi,
>> 
>> > Hi, thanks for review.
>> > 
>> > On Thu, Nov 25, 2010 at 01:58:04AM +, YAMAMOTO Takashi wrote:
>> >> hi,
>> >> 
>> >> - what's VM_PHYSSEG_OP_PG?
>> > 
>> > It's to lookup vm_physseg by "struct vm_page *", relying on that
>> > "struct vm_page *[]" is allocated linearly.  It'll be used to remove
>> > vm_page::phys_addr as we talked some time ago.
>> 
>> i'm not sure if commiting this unused uncommented code now helps it.
>> some try-and-benchmark cycles might be necessary given that
>> vm_page <-> paddr conversion could be performace critical.
> 
> If you really care performance, we can directly pass "struct vm_page
> *" to pmap_enter().
> 
> We're doing "struct vm_page *" -> "paddr_t" just before pmap_enter(),
> then doing "paddr_t" -> "vm_physseg" reverse lookup again in
> pmap_enter() to check if a given PA is managed.  What is really
> needed here is, to lookup "struct vm_page *" -> "vm_physseg" once
> and you'll know both paddr_t and managed or not.

i agree that the current code is not ideal in that respect.
otoh, i'm not sure if passing vm_physseg around is a good idea.

YAMAMOTO Takashi

> 
>> 
>> YAMAMOTO Takashi
> 
> -- 
> Masao Uebayashi / Tombi Inc. / Tel: +81-90-9141-4635


Re: CVS commit: src/sys/uvm

2010-11-25 Thread YAMAMOTO Takashi
[ adding cc: tech-kern@ ]

hi,

> On Wed, Nov 24, 2010 at 11:26:39PM -0800, Matt Thomas wrote:
>> 
>> On Nov 24, 2010, at 10:47 PM, Masao Uebayashi wrote:
>> 
>> > On Thu, Nov 25, 2010 at 05:44:21AM +, YAMAMOTO Takashi wrote:
>> >> hi,
>> >> 
>> >>> On Thu, Nov 25, 2010 at 04:18:25AM +, YAMAMOTO Takashi wrote:
>> >>>> hi,
>> >>>> 
>> >>>>> Hi, thanks for review.
>> >>>>> 
>> >>>>> On Thu, Nov 25, 2010 at 01:58:04AM +, YAMAMOTO Takashi wrote:
>> >>>>>> hi,
>> >>>>>> 
>> >>>>>> - what's VM_PHYSSEG_OP_PG?
>> >>>>> 
>> >>>>> It's to lookup vm_physseg by "struct vm_page *", relying on that
>> >>>>> "struct vm_page *[]" is allocated linearly.  It'll be used to remove
>> >>>>> vm_page::phys_addr as we talked some time ago.
>> >>>> 
>> >>>> i'm not sure if commiting this unused uncommented code now helps it.
>> >>>> some try-and-benchmark cycles might be necessary given that
>> >>>> vm_page <-> paddr conversion could be performace critical.
>> >>> 
>> >>> If you really care performance, we can directly pass "struct vm_page
>> >>> *" to pmap_enter().
>> >>> 
>> >>> We're doing "struct vm_page *" -> "paddr_t" just before pmap_enter(),
>> >>> then doing "paddr_t" -> "vm_physseg" reverse lookup again in
>> >>> pmap_enter() to check if a given PA is managed.  What is really
>> >>> needed here is, to lookup "struct vm_page *" -> "vm_physseg" once
>> >>> and you'll know both paddr_t and managed or not.
>> >> 
>> >> i agree that the current code is not ideal in that respect.
>> >> otoh, i'm not sure if passing vm_physseg around is a good idea.
>> > 
>> > It's great you share the interest.
>> > 
>> > I chose vm_physseg, because it was there.  I'm open to alternatives,
>> > but I don't think you have many options...
>> 
>> Passing vm_page * doesn't work if the page isn't managed since there
>> won't be a vm_page for the paddr_t.
>> 
>> Now passing both paddr_t and vm_page * would work and if the pointer
>> to the vm_page, it would be an unmanaged mapping.  This also gives the
>> access to mdpg without another lookup.
> 
> What if XIP'ed md(4), where physical pages are in .data (or .rodata)?
> 
> And don't forget that you're the one who first pointed out that
> allocating vm_pages for XIP is a pure waste of memory. ;)

i guess matt meant "if the pointer to the vm_page is NULL,".

> 
> I'm allocating vm_pages, only because of phys_addr and loan_count.
> I believe vm_pages is unnecessary for read-only XIP segments.
> Because they're read-only, and stateless.
> 
> I've already concluded that the current "managed or not" model
> doesn't work for XIP.  I'm pretty sure that my vm_physseg + off_t
> model can explain everything.  I'm rather waiting for a counter
> example how vm_physseg doesn't work...

i guess your suggestion is too vague.
where do you want to use vm_physseg * + off_t instead of vm_page * ?
getpages, pmap_enter, and?  how their function prototypes would be?
any valid paddr_t value will belong to exactly one vm_phsseg?

YAMAMOTO Takashi


Re: CVS commit: src/sys/kern

2010-12-19 Thread YAMAMOTO Takashi
hi,

> On Fri Dec 17 2010 at 22:34:04 +0000, YAMAMOTO Takashi wrote:
>> Module Name: src
>> Committed By:yamt
>> Date:Fri Dec 17 22:34:04 UTC 2010
>> 
>> Modified Files:
>>  src/sys/kern: vfs_lookup.c
>> 
>> Log Message:
>> - lookup_once: when crossing a mount point, don't keep the parent vnode 
>> locked.
>>   ie. don't lock a vnode while holding another vnode which belongs to a
>>   different filesystem.  otherwise we propagate slowness (or deadness) of a
>>   filesystem to another via vnode lock chain.
>> - lookup_parsepath: don't alter vnode states.  let the caller do it instead.
>> - add comments and assertions.
> 
> Hi, this breaks the state->dp != ndp->ni_dvp invariant in at least 3
> places in vfs_lookup.c:
> 
> http://www.gson.org/netbsd/bugs/build/build/2010.12.18.09.26.57/test.html#failed-tcs-summary
> 
> Can you have a look?

i reverted the change as i currently have no time to take a look.
i'll take a look later.  thanks.

YAMAMOTO Takashi

> 
> -- 
> dld karot toivorikkauttas, kyl rdtei ja lumpui piisaa


Re: CVS commit: src/sys/uvm

2010-12-21 Thread YAMAMOTO Takashi
hi,

> Could you ack this discussion?

sorry for dropping a ball.

> 
> On Tue, Dec 07, 2010 at 01:19:46AM +0900, Masao Uebayashi wrote:
>> On Thu, Nov 25, 2010 at 11:32:39PM +, YAMAMOTO Takashi wrote:
>> > [ adding cc: tech-kern@ ]
>> > 
>> > hi,
>> > 
>> > > On Wed, Nov 24, 2010 at 11:26:39PM -0800, Matt Thomas wrote:
>> > >> 
>> > >> On Nov 24, 2010, at 10:47 PM, Masao Uebayashi wrote:
>> > >> 
>> > >> > On Thu, Nov 25, 2010 at 05:44:21AM +0000, YAMAMOTO Takashi wrote:
>> > >> >> hi,
>> > >> >> 
>> > >> >>> On Thu, Nov 25, 2010 at 04:18:25AM +, YAMAMOTO Takashi wrote:
>> > >> >>>> hi,
>> > >> >>>> 
>> > >> >>>>> Hi, thanks for review.
>> > >> >>>>> 
>> > >> >>>>> On Thu, Nov 25, 2010 at 01:58:04AM +, YAMAMOTO Takashi wrote:
>> > >> >>>>>> hi,
>> > >> >>>>>> 
>> > >> >>>>>> - what's VM_PHYSSEG_OP_PG?
>> > >> >>>>> 
>> > >> >>>>> It's to lookup vm_physseg by "struct vm_page *", relying on that
>> > >> >>>>> "struct vm_page *[]" is allocated linearly.  It'll be used to 
>> > >> >>>>> remove
>> > >> >>>>> vm_page::phys_addr as we talked some time ago.
>> > >> >>>> 
>> > >> >>>> i'm not sure if commiting this unused uncommented code now helps 
>> > >> >>>> it.
>> > >> >>>> some try-and-benchmark cycles might be necessary given that
>> > >> >>>> vm_page <-> paddr conversion could be performace critical.
>> > >> >>> 
>> > >> >>> If you really care performance, we can directly pass "struct vm_page
>> > >> >>> *" to pmap_enter().
>> > >> >>> 
>> > >> >>> We're doing "struct vm_page *" -> "paddr_t" just before 
>> > >> >>> pmap_enter(),
>> > >> >>> then doing "paddr_t" -> "vm_physseg" reverse lookup again in
>> > >> >>> pmap_enter() to check if a given PA is managed.  What is really
>> > >> >>> needed here is, to lookup "struct vm_page *" -> "vm_physseg" once
>> > >> >>> and you'll know both paddr_t and managed or not.
>> > >> >> 
>> > >> >> i agree that the current code is not ideal in that respect.
>> > >> >> otoh, i'm not sure if passing vm_physseg around is a good idea.
>> > >> > 
>> > >> > It's great you share the interest.
>> > >> > 
>> > >> > I chose vm_physseg, because it was there.  I'm open to alternatives,
>> > >> > but I don't think you have many options...
>> > >> 
>> > >> Passing vm_page * doesn't work if the page isn't managed since there
>> > >> won't be a vm_page for the paddr_t.
>> > >> 
>> > >> Now passing both paddr_t and vm_page * would work and if the pointer
>> > >> to the vm_page, it would be an unmanaged mapping.  This also gives the
>> > >> access to mdpg without another lookup.
>> > > 
>> > > What if XIP'ed md(4), where physical pages are in .data (or .rodata)?
>> > > 
>> > > And don't forget that you're the one who first pointed out that
>> > > allocating vm_pages for XIP is a pure waste of memory. ;)
>> > 
>> > i guess matt meant "if the pointer to the vm_page is NULL,".
>> > 
>> > > 
>> > > I'm allocating vm_pages, only because of phys_addr and loan_count.
>> > > I believe vm_pages is unnecessary for read-only XIP segments.
>> > > Because they're read-only, and stateless.
>> > > 
>> > > I've already concluded that the current "managed or not" model
>> > > doesn't work for XIP.  I'm pretty sure that my vm_physseg + off_t
>> > > model can explain everything.  I'm rather waiting for a counter
>> > > example how vm_physseg doesn't work...
>> > 
>> > i guess your suggestion is too vague.
>> > where do you want to use vm_physseg * + off_t instead of vm_page * ?
>> > getpages, pmap_enter, and?  how their function prototypes would be?
>> 
>> The basic idea is straightforward; always allocate vm_physseg for
>> memories/devices.  If a vm_physseg is used as general purpose
>> memory, you allocate vm_page[] (as vm_physseg::pgs).  If it's
>> potentially mapped as cached, you allocate pvh (as vm_physseg:pvh).

can you provide function prototypes?

>> 
>> Keep vm_physseg * + off_t array on stack.  If UVM objects uses
>> vm_page (e.g. vnode), its pager looks up vm_page -> vm_physseg *
>> + off_t *once* and cache it on stack.

do you mean something like this?
struct {
vm_physseg *hoge;
off_t fuga;
} foo [16];

YAMAMOTO Takashi

>> 
>> > any valid paddr_t value will belong to exactly one vm_phsseg?
>> 
>> That's the idea.  This would clarify mem(4) backend too.
>> 
>> Note that allocating vm_physseg for device segments is cheap.


Re: CVS commit: src/sys/uvm

2011-01-04 Thread YAMAMOTO Takashi
hi,

> I take silence as "no objection".

the silence in this case means was-busy-for-other-things-and-forgot.
sorry.

>> I have no real code for this big picture at this moment.  Making
>> vm_physseg available as reference is the first step.  This only
>> changes uvm_page_physload() to return a pointer:
>> 
>>  -void uvm_page_physload();
>>  +void *uvm_page_physload();
>> 
>> But this makes XIP pager MUCH cleaner.  The reason has been explained
>> many times.

because the separate uvm_page_physload_device is no longer necessary,
you mean?  i have no problem with the step.

>> Making fault handlers and pagers to use vm_physseg * + off_t is
>> the next step, and I don't intend to work on it now.  I just want
>> to explain the big picture.
>> 
>> > 
>> > >> 
>> > >> Keep vm_physseg * + off_t array on stack.  If UVM objects uses
>> > >> vm_page (e.g. vnode), its pager looks up vm_page -> vm_physseg *
>> > >> + off_t *once* and cache it on stack.
>> > 
>> > do you mean something like this?
>> >struct {
>> >vm_physseg *hoge;
>> >off_t fuga;
>> >} foo [16];
>> 
>> Yes.
>> 
>> Or cache vm_page * with it, like:
>> 
>>  struct vm_id {
>>  vm_physseg *seg;
>>  off_t off;
>>  vm_page *pg;
>>  };
>> 
>>  uvm_fault()
>>  {
>>  vm_id pgs[];
>>  :
>>  }
>> 
>> Vnode pager (genfs_getpages) takes vm_page's by looking up
>> vnode::v_uobj's list, or uvn_findpages().
>> 
>> When it returns back to fault handler, we have to lookup vm_physseg
>> for each page.  Then fill the "seg" slot above (assuming we'll
>> remove vm_page::phys_addr soon).
>> 
>> Fault handler calls per-vm_page operations iff vm_page slot is filled.
>> XIP pages are not pageq'ed.

pgo_get returns either seg+off or pg for each vm_id slots?

>> XIP pages don't need vm_page, but
>> cached because it's vnode.

can you explain this sentence?

>> (Just in case, have you read my paper?)

which paper?  i guess no.

YAMAMOTO Takashi


Re: CVS commit: src/tests/fs/common

2011-01-05 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: pooka
> Date: Fri Dec 31 18:16:41 UTC 2010
> 
> Modified Files:
>   src/tests/fs/common: h_fsmacros.h
> 
> Log Message:
> Introduce r/o tests.  They do two mounts: the first one is r/w and
> runs a generator which primes the fs.  The second one is r/o and
> does the actual testing.  Also, introduce a nfsro fstype which does
> a clientside r/w mount for a r/o server export.
> 
> requested by yamt

thanks.

> (one nfsro test currently fails with EROFS vs. EACCES.  Hopefully
> someone else can debate the correct errno)

the NFS ACCESS procedure, which is used for open time permission checks,
does not have a way to distinguish EROFS and EACCES.  ie. EACCES is
the expected behaviour in this case.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.23 -r1.24 src/tests/fs/common/h_fsmacros.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/kern

2011-01-10 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Sat Jan  8 20:37:05 UTC 2011
> 
> Modified Files:
>   src/sys/kern: vfs_wapbl.c
> 
> Log Message:
> Add two sysctls one that does verbose transaction logging and a second one
> that disables flushing the disk cache (which is fast but dangerous for
> data integrity). From simon a long while back.

i don't think this kind of knob should be system global.
are they merely for debug?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.38 -r1.39 src/sys/kern/vfs_wapbl.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/tests/fs/common

2011-01-10 Thread YAMAMOTO Takashi
hi,

> On Thu, Jan 06, 2011 at 12:53:28AM +0000, YAMAMOTO Takashi wrote:
>  > > (one nfsro test currently fails with EROFS vs. EACCES.  Hopefully
>  > > someone else can debate the correct errno)
>  > 
>  > the NFS ACCESS procedure, which is used for open time permission checks,
>  > does not have a way to distinguish EROFS and EACCES.  ie. EACCES is
>  > the expected behaviour in this case.
> 
> Shouldn't the nfs client know that it's supposed to be readonly?

probably.  please feel free to fix it. :-)

however,

> Or is
> this a case where the nfs client is read-write but the server isn't?

i think so, yes.  in this case, the client can't know EROFS at open time.

YAMAMOTO Takashi

> 
> -- 
> David A. Holland
> dholl...@netbsd.org


Re: CVS commit: src/sys/uvm

2011-01-16 Thread YAMAMOTO Takashi
hi,

>> >> (Just in case, have you read my paper?)
>> 
>> which paper?  i guess no.
> 
> http://uebayasi.dyndns.org/~uebayasi/tmp/xip.pdf

i will put it on my todo list.  thanks.

YAMAMOTO Takashi


Re: CVS commit: src/sys/uvm

2011-01-20 Thread YAMAMOTO Takashi
hi,

do you have any plan to use pg->offset of anon pages,
or is it an unnecessary side-effect?

YAMAMOTO Takashi

> Module Name:  src
> Committed By: matt
> Date: Tue Jan  4 08:26:33 UTC 2011
> 
> Modified Files:
>   src/sys/uvm: uvm_extern.h uvm_fault.c uvm_km.c uvm_page.c
> 
> Log Message:
> Add better color matching selecting free pages.  KM pages will now allocated
> so that VA and PA have the same color.  On a page fault, choose a physical
> page that has the same color as the virtual address.
> 
> When allocating kernel memory pages, allow the MD to specify a preferred
> VM_FREELIST from which to choose pages.  For machines with large amounts
> of memory (> 4GB), all kernel memory to come from <4GB to reduce the amount
> of bounce buffering needed with 32bit DMA devices.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.167 -r1.168 src/sys/uvm/uvm_extern.h
> cvs rdiff -u -r1.178 -r1.179 src/sys/uvm/uvm_fault.c
> cvs rdiff -u -r1.106 -r1.107 src/sys/uvm/uvm_km.c
> cvs rdiff -u -r1.168 -r1.169 src/sys/uvm/uvm_page.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/share/man/man9

2011-01-25 Thread YAMAMOTO Takashi
hi,

> How about ABI?  KASSERT(9) should preserve ABI IMO.

sorry, i'm not sure what you mean.  can you explain?

YAMAMOTO Takashi


Re: CVS commit: src/lib/libc/time

2011-02-11 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Thu Jan  6 02:41:34 UTC 2011
> 
> Modified Files:
>   src/lib/libc/time: localtime.c
> 
> Log Message:
> Since localsub and gmtsub are called recursively to search for the local
> time, setting EOVERFLOW at the inmost level will unfortunately persist,
> even if later calls to those functions succeed. Move the EOVERFLOW setting
> to the top level calls.

did you forget to update gmtime/gmtime_r/offtime/offtime_r?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.50 -r1.51 src/lib/libc/time/localtime.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libc/time

2011-02-13 Thread YAMAMOTO Takashi
hi,

> On Feb 11, 11:47pm, y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
> -- Subject: Re: CVS commit: src/lib/libc/time
> 
> | hi,
> | 
> | > Module Name:  src
> | > Committed By: christos
> | > Date: Thu Jan  6 02:41:34 UTC 2011
> | > 
> | > Modified Files:
> | >   src/lib/libc/time: localtime.c
> | > 
> | > Log Message:
> | > Since localsub and gmtsub are called recursively to search for the local
> | > time, setting EOVERFLOW at the inmost level will unfortunately persist,
> | > even if later calls to those functions succeed. Move the EOVERFLOW setting
> | > to the top level calls.
> | 
> | did you forget to update gmtime/gmtime_r/offtime/offtime_r?
> 
> What do you mean? These call gmtsub directly...

and gmtsub doesn't set errno for them anymore.

YAMAMOTO Takashi

> 
> christos


Re: CVS commit: src/lib/librumphijack

2011-02-21 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: pooka
> Date: Mon Feb  7 19:34:39 UTC 2011
> 
> Modified Files:
>   src/lib/librumphijack: hijack.c
> 
> Log Message:
> Force gcc to generate a stack frame for the call to dlsym(RTLD_NEXT).
> Without this hack at least amd64 -O2 just used jmp and The Wrong
> Thing happened.

do you want -fno-optimize-sibling-calls ?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.31 -r1.32 src/lib/librumphijack/hijack.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/librumphijack

2011-02-22 Thread YAMAMOTO Takashi
> On Mon Feb 21 2011 at 23:19:47 +0000, YAMAMOTO Takashi wrote:
>> > Module Name:   src
>> > Committed By:  pooka
>> > Date:  Mon Feb  7 19:34:39 UTC 2011
>> > 
>> > Modified Files:
>> >src/lib/librumphijack: hijack.c
>> > 
>> > Log Message:
>> > Force gcc to generate a stack frame for the call to dlsym(RTLD_NEXT).
>> > Without this hack at least amd64 -O2 just used jmp and The Wrong
>> > Thing happened.
>> 
>> do you want -fno-optimize-sibling-calls ?
> 
> Without doing a test-compile, looks like it.  I guess I'll have to move
> that function to its own source module, or is there a way to enable
> it per-function?

i don't think there's a way for our version of gcc to make it per-function.
later versions have "optimize" function attribute.
the real fix here would be an attibute for dlsym, not for a caller of it, tho.

YAMAMOTO Takashi

> 
> -- 
> dld karot toivorikkauttas, kyl rdtei ja lumpui piisaa


Re: CVS commit: src

2011-02-22 Thread YAMAMOTO Takashi
hi,

> On Feb 22, 2011, at 1:31 PM, YAMAMOTO Takashi wrote:
> 
>> Module Name: src
>> Committed By:yamt
>> Date:Tue Feb 22 21:31:16 UTC 2011
>> 
>> Added Files:
>>  src/common/lib/libc/gen: radixtree.c
>>  src/sys/sys: radixtree.h
>> 
>> Log Message:
>> an implementation of radix tree.  the idea from linux.
> 
> How is that different from ptree?

while i'm not familiar with ptree...

- there is no "node" structure which needs to be embedded into user structures.
- tagging functionality.

i plan to use it for page cache.  (as linux does)
mainly for smaller vm_page and faster fsync.

YAMAMOTO Takashi


Re: CVS commit: src

2011-02-23 Thread YAMAMOTO Takashi
hi,

> On Wed, Feb 23, 2011 at 01:33:05AM +0000, YAMAMOTO Takashi wrote:
>> hi,
>> 
>> > On Feb 22, 2011, at 1:31 PM, YAMAMOTO Takashi wrote:
>> > 
>> >> Module Name:  src
>> >> Committed By: yamt
>> >> Date: Tue Feb 22 21:31:16 UTC 2011
>> >> 
>> >> Added Files:
>> >>   src/common/lib/libc/gen: radixtree.c
>> >>   src/sys/sys: radixtree.h
>> >> 
>> >> Log Message:
>> >> an implementation of radix tree.  the idea from linux.
>> > 
>> > How is that different from ptree?
>> 
>> while i'm not familiar with ptree...
>> 
>> - there is no "node" structure which needs to be embedded into user 
>> structures.
>> - tagging functionality.
> 
> Aren't both ptree and radixtree missing documentation?

right.  both of them are work-in-progress piece of code.  well, at least
radixtree is.
are you suggesting something?

YAMAMOTO Takashi

> 
> Dave
> 
> -- 
> David Young OJC Technologies
> dyo...@ojctech.com  Urbana, IL * (217) 344-0444 x24


Re: CVS commit: src

2011-02-23 Thread YAMAMOTO Takashi
hi,

> On Thu, Feb 24, 2011 at 12:12:50AM +0000, YAMAMOTO Takashi wrote:
>> I wrote:
>> > Aren't both ptree and radixtree missing documentation?
>> 
>> right.  both of them are work-in-progress piece of code.  well, at least
>> radixtree is.
>> are you suggesting something?
> 
> Yes: commit new libraries with documentation.

sorry, i don't volunteer to write documentation for work-in-progress
code which will likely change.  do you want me remove it for now?
are you happier if it was on a dedicated branch?

YAMAMOTO Takashi

> 
> Dave
> 
> -- 
> David Young OJC Technologies
> dyo...@ojctech.com  Urbana, IL * (217) 344-0444 x24


Re: CVS commit: src

2011-03-01 Thread YAMAMOTO Takashi
hi,

> Hello,
> 
> y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
>> >> Log Message:
>> >> an implementation of radix tree.  the idea from linux.
>> > 
>> > How is that different from ptree?
>> 
>> while i'm not familiar with ptree...
>> 
> 
> Patricia/Radix tree added by Matt (see src/common/lib/libc/gen/ptree.c
> and sys/ptree.h).  So now there are three Radix tree variations (each
> lack documentation, heh).
> 
>> - there is no "node" structure which needs to be embedded into user
>> structures.
>> - tagging functionality.
>> 
>> i plan to use it for page cache.  (as linux does)
>> mainly for smaller vm_page and faster fsync.
> 
> Couple years ago you have added Radix Priority Search Tree (RPST).
> Do you plan to use it somewhere as well?

i was vaguely thinking to use it for p->v lookup when i wrote it.

i don't have an immediate use plan at this point.

i didn't plug it to Makefile because a future use case might need
abi-incompatible changes.

YAMAMOTO Takashi

> 
> Thanks.
> 
> -- 
> Mindaugas


Re: CVS commit: src/sys/kern

2011-03-15 Thread YAMAMOTO Takashi
hi,

> On 12.03.11 08:16, YAMAMOTO Takashi wrote:
>> Module Name: src
>> Committed By:yamt
>> Date:Sat Mar 12 07:16:50 UTC 2011
> 
> Hi, nice to read/hear you're alive.
> 
> Christoph

thank you.

YAMAMOTO Takashi


Re: CVS commit: src/sys/netinet

2011-04-14 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: martin
> Date: Sat Apr  9 20:34:36 UTC 2011
> 
> Modified Files:
>   src/sys/netinet: ip_output.c
> 
> Log Message:
> We do not do checksums on loopback interfaces, not even if fragmenting.
> Fixes PR kern/43664.

please just weaken the assertion and clear the flag,
rather than complicating the code.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.205 -r1.206 src/sys/netinet/ip_output.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys

2011-04-24 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: sborrill
> Date: Fri Apr  8 13:56:51 UTC 2011
> 
> Modified Files:
>   src/sys/dev/pci: if_alc.c if_bge.c
>   src/sys/net: if_vlan.c
> 
> Log Message:
> PR kern/38871
> 
> Fix LAN on bge(4), alc(4). Flag VLAN capability in ec_capenable as used by 
> network
> card drivers.

- who clears the flag?

- probably it would be better to have separate flags for tx and rx.
  (i thought ETHERCAP_VLAN_HWTAGGING was for transmitting,
  but our tree seems inconsistent.)

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.2 -r1.3 src/sys/dev/pci/if_alc.c
> cvs rdiff -u -r1.190 -r1.191 src/sys/dev/pci/if_bge.c
> cvs rdiff -u -r1.66 -r1.67 src/sys/net/if_vlan.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/netinet

2011-04-25 Thread YAMAMOTO Takashi
hi,

> On Thu, Apr 14, 2011 at 07:03:49AM +0000, YAMAMOTO Takashi wrote:
>> please just weaken the assertion and clear the flag,
>> rather than complicating the code.
> 
> I'm not quite sure I see exactly what you would like the code to look like.
> What we have now:
> 
> /*
>  * We may not use checksums on loopback interfaces
>  */
> if (__predict_false(ifp == NULL) ||
> IN_NEED_CHECKSUM(ifp, M_CSUM_IPv4)) {
> if (sw_csum & M_CSUM_IPv4) {
> ip->ip_sum = in_cksum(m, hlen);
> m->m_pkthdr.csum_flags &= ~M_CSUM_IPv4;
> } else {
> KASSERT(m->m_pkthdr.csum_flags & M_CSUM_IPv4);
> KASSERT(M_CSUM_DATA_IPv4_IPHL(m->m_pkthdr.csum_data) 
> >=
> sizeof(struct ip));
> }
> }
> 
> This could be refactored into:
> 
> if (sw_csum & M_CSUM_IPv4) {
> ip->ip_sum = in_cksum(m, hlen);
> m->m_pkthdr.csum_flags &= ~M_CSUM_IPv4;
> } else if (__predict_false(ifp == NULL)
>   || IN_NEED_CHECKSUM(ifp, M_CSUM_IPv4)) {
> KASSERT(m->m_pkthdr.csum_flags & M_CSUM_IPv4);
> KASSERT(M_CSUM_DATA_IPv4_IPHL(m->m_pkthdr.csum_data) 
> >=
> sizeof(struct ip));
>   }
>     }
> 
> or a variant that embeds the else if condition into both KASSERTs, which seems
> pretty ugly to me.
> 
> Can you please clarify?

somthing like the following.

YAMAMOTO Takashi

Index: ip_output.c
===
RCS file: /cvsroot/src/sys/netinet/ip_output.c,v
retrieving revision 1.208
diff -u -p -r1.208 ip_output.c
--- ip_output.c 14 Apr 2011 15:53:36 -  1.208
+++ ip_output.c 25 Apr 2011 22:41:43 -
@@ -1010,12 +1010,19 @@ ip_fragment(struct mbuf *m, struct ifnet
m->m_pkthdr.len = mhlen + len;
m->m_pkthdr.rcvif = (struct ifnet *)0;
mhip->ip_sum = 0;
+   KASSERT((m->m_pkthdr.csum_flags & M_CSUM_IPv4) == 0);
if (sw_csum & M_CSUM_IPv4) {
mhip->ip_sum = in_cksum(m, mhlen);
-   KASSERT((m->m_pkthdr.csum_flags & M_CSUM_IPv4) == 0);
} else {
-   m->m_pkthdr.csum_flags |= M_CSUM_IPv4;
+   /*
+* checksum is hw-offloaded or not necessary.
+*/
+   m->m_pkthdr.csum_flags |=
+   m0->m_pkthdr.csum_flags & M_CSUM_IPv4;
m->m_pkthdr.csum_data |= mhlen << 16;
+   KASSERT(!(ifp != NULL &&
+   IN_NEED_CHECKSUM(ifp, M_CSUM_IPv4))
+   || (m->m_pkthdr.csum_flags & M_CSUM_IPv4) != 0);
}
IP_STATINC(IP_STAT_OFRAGMENTS);
fragments++;
@@ -1030,19 +1037,17 @@ ip_fragment(struct mbuf *m, struct ifnet
ip->ip_len = htons((u_int16_t)m->m_pkthdr.len);
ip->ip_off |= htons(IP_MF);
ip->ip_sum = 0;
-   /*
-* We may not use checksums on loopback interfaces
-*/
-   if (__predict_false(ifp == NULL) ||
-   IN_NEED_CHECKSUM(ifp, M_CSUM_IPv4)) {
-   if (sw_csum & M_CSUM_IPv4) {
-   ip->ip_sum = in_cksum(m, hlen);
-   m->m_pkthdr.csum_flags &= ~M_CSUM_IPv4;
-   } else {
-   KASSERT(m->m_pkthdr.csum_flags & M_CSUM_IPv4);
-   KASSERT(M_CSUM_DATA_IPv4_IPHL(m->m_pkthdr.csum_data) >=
-   sizeof(struct ip));
-   }
+   if (sw_csum & M_CSUM_IPv4) {
+   ip->ip_sum = in_cksum(m, hlen);
+   m->m_pkthdr.csum_flags &= ~M_CSUM_IPv4;
+   } else {
+   /*
+* checksum is hw-offloaded or not necessary.
+*/
+   KASSERT(!(ifp != NULL && IN_NEED_CHECKSUM(ifp, M_CSUM_IPv4))
+  || (m->m_pkthdr.csum_flags & M_CSUM_IPv4) != 0);
+   KASSERT(M_CSUM_DATA_IPv4_IPHL(m->m_pkthdr.csum_data) >=
+   sizeof(struct ip));
}
 sendorfree:
/*


Re: CVS commit: src/sys/net

2011-06-23 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: kefren
> Date: Tue Jun 21 14:30:20 UTC 2011
> 
> Modified Files:
>   src/sys/net: if_mpls.c
> 
> Log Message:
> learn mpls interface how to prepend multiple shims by using a vector of
> smpls_addrs in sockaddr_mpls. The number of smpls_addrs is found from
> smpls_len. First label encountered is BoS.
> XXX: need to do the same for LSE and this feature needs to be documented.

what will use this?  (just curious)

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.5 -r1.6 src/sys/net/if_mpls.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/uvm

2011-06-27 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: hannken
> Date: Mon Jun 27 15:56:37 UTC 2011
> 
> Modified Files:
>   src/sys/uvm: uvm_amap.c
> 
> Log Message:
> amap_copy(): Keep the source amap locked until its lock has been copied.

btw, this code seems to assume that uvm_anfree does not release the lock
even temporarily while the comment on uvm_anfree1 says the opposite.

YAMAMOTO Takashi

> 
> Kernel assertion "anon->an_lock == amap->am_lock" no longer fails.
> 
> Ok: Mindaugas Rasiukevicius 
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.99 -r1.100 src/sys/uvm/uvm_amap.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/uvm

2011-07-05 Thread YAMAMOTO Takashi
hi,

> y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
>> > Module Name:   src
>> > Committed By:  hannken
>> > Date:  Mon Jun 27 15:56:37 UTC 2011
>> > 
>> > Modified Files:
>> >src/sys/uvm: uvm_amap.c
>> > 
>> > Log Message:
>> > amap_copy(): Keep the source amap locked until its lock has been copied.
>> 
>> btw, this code seems to assume that uvm_anfree does not release the lock
>> even temporarily while the comment on uvm_anfree1 says the opposite.
> 
> http://www.netbsd.org/~rmind/uvm_anon_freelst.diff
> 
> Looks good?

i don't understand what it solves.  can you explain a little?

YAMAMOTO Takashi

> 
>> 
>> YAMAMOTO Takashi
> 
> -- 
> Mindaugas


Re: CVS commit: src/sys/uvm

2011-07-25 Thread YAMAMOTO Takashi
hi,

> y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
>> >> > Log Message:
>> >> > amap_copy(): Keep the source amap locked until its lock has been
>> >> > copied.
>> >> 
>> >> btw, this code seems to assume that uvm_anfree does not release the
>> >> lock even temporarily while the comment on uvm_anfree1 says the
>> >> opposite.
>> > 
>> > http://www.netbsd.org/~rmind/uvm_anon_freelst.diff
>> > 
>> > Looks good?
>> 
>> i don't understand what it solves.  can you explain a little?
> 
> Here is the updated patch after your changes:
> 
> http://www.netbsd.org/~rmind/uvm_anon_freelst2.diff
> 
> As you noted, uvm_anfree() can temporarily release the amap lock - that can
> happen in amap_copy().  Patch closes the race by moving uvm_anfree() further,
> and changes the semantics of the function, now called uvm_anon_freelst(), to
> return with amap lock released (plus free anons without lock held).

the temporary release of the amap lock is only for O->A loan
which you disabled, isn't it?

YAMAMOTO Takashi

> 
> -- 
> Mindaugas


Re: CVS commit: src/sys/kern

2011-08-09 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: dholland
> Date: Tue Aug  9 23:46:05 UTC 2011
> 
> Modified Files:
>   src/sys/kern: vfs_lookup.c
> 
> Log Message:
> Fail namei immediately if searchdir is unlinked / has been rmdir'd.
> Do this by checking if v_size == 0. Should fix PR 44658 (and PR 32661).

why is this necessary?  can't we just let VOP_LOOKUP fail?

the v_size == 0 check sounds wrong.  does it work for eg. nfs?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.186 -r1.187 src/sys/kern/vfs_lookup.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/kern

2011-08-10 Thread YAMAMOTO Takashi
> On Wed, Aug 10, 2011 at 03:10:13AM +0000, YAMAMOTO Takashi wrote:
>  > > Log Message:
>  > > Fail namei immediately if searchdir is unlinked / has been rmdir'd.
>  > > Do this by checking if v_size == 0. Should fix PR 44658 (and PR 32661).
>  > 
>  > why is this necessary?  can't we just let VOP_LOOKUP fail?
> 
> Not to fix PR 44658.

it's better to fix the vn_isunder check instead of avoiding running it.
IMO vn_isunder should return acutal error code (eg. ENOENT) rather
than just a boolean so that callers can decide what to do.

> 
>  > the v_size == 0 check sounds wrong.  does it work for eg. nfs?
> 
> It apparently does break nullfs, so I've reverted it.
> 
> Is there any way to check this correctly/safely above the filesystem?

if "above the filesystem" means "without calling VOPs", i don't think
there's a way.

YAMAMOTO Takashi

> 
> -- 
> David A. Holland
> dholl...@netbsd.org


Re: CVS commit: src

2011-08-10 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: manu
> Date: Wed Aug  3 04:11:17 UTC 2011
> 
> Modified Files:
>   src/bin/cp: cp.c utils.c
>   src/bin/mv: mv.c
>   src/distrib/sets/lists/comp: mi
>   src/lib/libc/gen: Makefile.inc extattr.3 extattr.c
>   src/lib/libc/sys: extattr_get_file.2
>   src/sys/sys: extattr.h
> 
> Log Message:
> Make cp -p and mv preverve extended attributes, and complain if they cannot.
> 
> Also introduce library functions for copying extended attributes from one
> file to another:
> - extattr_copy_file, extattr_copy_fd, extattr_copy_link, with FreeBSD style,
>   where a namespace is to be supplied
> - cpxattr, fcpxattr, lcpxattr, with Linux style, where all namespaces
>   accessible to the caller are copied, and the others are silently ignored.

is extattr_namespace_access really necessary?
uid-based priviledge check in userland is often a mistake.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.55 -r1.56 src/bin/cp/cp.c
> cvs rdiff -u -r1.39 -r1.40 src/bin/cp/utils.c
> cvs rdiff -u -r1.41 -r1.42 src/bin/mv/mv.c
> cvs rdiff -u -r1.1651 -r1.1652 src/distrib/sets/lists/comp/mi
> cvs rdiff -u -r1.178 -r1.179 src/lib/libc/gen/Makefile.inc
> cvs rdiff -u -r1.3 -r1.4 src/lib/libc/gen/extattr.3
> cvs rdiff -u -r1.2 -r1.3 src/lib/libc/gen/extattr.c
> cvs rdiff -u -r1.3 -r1.4 src/lib/libc/sys/extattr_get_file.2
> cvs rdiff -u -r1.6 -r1.7 src/sys/sys/extattr.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src

2011-08-10 Thread YAMAMOTO Takashi
hi,

> On Wed, Aug 10, 2011 at 08:59:48AM +0000, YAMAMOTO Takashi wrote:
>> is extattr_namespace_access really necessary?
>> uid-based priviledge check in userland is often a mistake.
> 
> For now it duplicates the same simple access check as in kernel: 
> system attributes are restricted to root. This is just a helper function,
> it is not exported. I immagine it could move to kernel when we introduce
> more namespaces with different acces semantics. But we are not there yet.

what's wrong with just letting the kernel decide and handle EPERM?

YAMAMOTO Takashi

> 
> -- 
> Emmanuel Dreyfus
> m...@netbsd.org


Re: CVS commit: src/sys/uvm

2011-08-10 Thread YAMAMOTO Takashi
[ moving to tech-kern ]

hi,

> y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
>> > 
>> > Here is the updated patch after your changes:
>> > 
>> > http://www.netbsd.org/~rmind/uvm_anon_freelst2.diff
>> > 
>> > As you noted, uvm_anfree() can temporarily release the amap lock - that
>> > can happen in amap_copy().  Patch closes the race by moving uvm_anfree
>> > () further, and changes the semantics of the function, now called
>> > uvm_anon_freelst(), to return with amap lock released (plus free anons
>> > without lock held).
>> 
>> the temporary release of the amap lock is only for O->A loan
>> which you disabled, isn't it?
> 
> Right, uvm_anon_locklaonpg() dance can happen only in O->A case.  However,
> having uvm_anfree() able to release the lock by its interface definition
> is potentially defective.  It is the main motivation why I want to slightly
> rework the code into uvm_anon_freelst() which would always drop the lock
> and move freeing of anons to the end point.  Cleaner, less error prone.

the committed change seems broken in case uvm_anon_dispose sets PG_RELEASED.
in that case, uvm_anon_freelst should leave the anon as it will be freed by
uvm_anon_release later.

YAMAMOTO Takashi

> 
> -- 
> Mindaugas


Re: CVS commit: src/sys/fs/puffs

2011-09-22 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: manu
> Date: Wed Sep 21 15:36:33 UTC 2011
> 
> Modified Files:
>   src/sys/fs/puffs: puffs_vfsops.c puffs_vnops.c
> 
> Log Message:
> Make sure ioflush does not sleep in PUFFS code path, waiting for a mutex,
> a memory allocation, or a response from the filesystem.
> 
> This avoids deadlocks in the following situations:
> 1) when memory is low: ioflush waits the fileystem, the fielsystem waits
>for memory

can you explain how it is a problem?

YAMAMOTO Takashi

> 2) when the filesystem does not respond (e.g.: network outage ona
>distributed filesystem)
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.96 -r1.97 src/sys/fs/puffs/puffs_vfsops.c
> cvs rdiff -u -r1.155 -r1.156 src/sys/fs/puffs/puffs_vnops.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys

2011-09-22 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: manu
> Date: Fri Sep 23 01:57:32 UTC 2011
> 
> Modified Files:
>   src/sys/fs/puffs: puffs_vnops.c
>   src/sys/miscfs/syncfs: sync_subr.c
> 
> Log Message:
> Fix the build that was broken by struct lwp *updateproc reference in
> RUMP-visible code. Instead of checking that updateproc (aka ioflush,
> aka syncer) will not sleep in PUFFS code, I check for any kernel thread:
> after all none of them are designed to hang awaiting for a remote filesystem
> operation to complete.

i don't think it's a good idea to restrict what kernel threads can do
in this way.  please revert.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.156 -r1.157 src/sys/fs/puffs/puffs_vnops.c
> cvs rdiff -u -r1.46 -r1.47 src/sys/miscfs/syncfs/sync_subr.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/fs/puffs

2011-09-23 Thread YAMAMOTO Takashi
hi,

> YAMAMOTO Takashi  wrote:
> 
>> > This avoids deadlocks in the following situations:
>> > 1) when memory is low: ioflush waits the fileystem, the fielsystem waits
>> >for memory 
>> can you explain how it is a problem?
> 
> As I understand, one way to free memory is to flush vnode backed pages
> to the backend storage. If ioflush calls VOP_FSYNC on a memory-starved
> userland filesystem, it will get stuck until the filesystem gets memory
> again, and while it is stuck, it does not help freeing memory.

as i told you a few times, ioflush is not a thread to free memory.
pagedaemon is.  please read ufs_bmaparray and grep "uvm.pagedaemon_lwp"
in src/sys/kern/ to see what other filesystems do.  i don't think this
approach works for puffs because it's almost impossible to say in which
cases an operation needs memory allocation to complete, though.

YAMAMOTO Takashi

> 
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org


Re: CVS commit: src/sys/fs/puffs

2011-09-23 Thread YAMAMOTO Takashi
hi,

> YAMAMOTO Takashi  wrote:
> 
>> as i told you a few times, ioflush is not a thread to free memory.
>> pagedaemon is. 
> 
> Sure ioflush do not directly free memory, but vnodes' dirty page use
> memory, don't they? If ioflush stops working, is pageadaemon able to
> pageout that kind of memory?

pagedaemon flushes dirty pages by itself, yes.

> 
>> please read ufs_bmaparray and grep "uvm.pagedaemon_lwp"
>> in src/sys/kern/ to see what other filesystems do.  i don't think this
>> approach works for puffs because it's almost impossible to say in which
>> cases an operation needs memory allocation to complete, though.
> 
> It is indeed impossible, as we cannot know what the userland filesystem
> will do.

a possible solution would be local page recycling.  ie. reserve some pages
and put them onto a page queue dedicated for a given set of processes
so that pages can be recycled in the set independently from the global queue.

YAMAMOTO Takashi

> 
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org


Re: CVS commit: src/sys/fs/puffs

2011-10-11 Thread YAMAMOTO Takashi
hi,

> YAMAMOTO Takashi  wrote:
> 
>> > Sure ioflush do not directly free memory, but vnodes' dirty page use
>> > memory, don't they? If ioflush stops working, is pageadaemon able to
>> > pageout that kind of memory? 
>> pagedaemon flushes dirty pages by itself, yes.
> 
> So this is not a problem to get ioflush traped forever awaiting for a
> VOP_FSYNC completion? There must be some drawback, otherwise ioflush
> would be useless.

if it trapped forever, it's a bug and should be fixed.  my point was
that your change didn't fix the bug.  blocking ioflush is merely a symptom.

on the other hand, it needlessly introduced a limitation on what kernel
threads can do.  now we have vdrain_thread which i think can hit the
LW_SYSTEM assertion in puffs_vnop_fsync.

> 
>> a possible solution would be local page recycling.  ie. reserve some pages
>> and put them onto a page queue dedicated for a given set of processes
>> so that pages can be recycled in the set independently from the global queue.
> 
> But we do not know how much memory the userland fileserver is going to
> require. We can have some reserves, but we must be ready to kill pigs if
> too much memory is retained by the fileserver. HEAD does it, but
> netbsd-5 prefers to hang.

you don't need to know how much memory the server will use.
when the server runs out of the reserved set of pages, you can reuse
pages in the set.

YAMAMOTO Takashi

> 
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org


Re: CVS commit: src/sys/fs/puffs

2011-10-11 Thread YAMAMOTO Takashi
hi,

> YAMAMOTO Takashi  wrote:
> 
>> if it trapped forever, it's a bug and should be fixed.  my point was
>> that your change didn't fix the bug.  blocking ioflush is merely a symptom.
> 
> The problem with userland filesystems is that we may have little control
> as theses may be third pary programs. Should kernel threads trust theses
> processes in order to run as intended?

i guess it depends.

is ioflush blocking forever worse than a userland application
blocking forever?  it depends.

even if it's a problem, it isn't a problem specific to userland filesystems.
filesystems have varying performance and reliability.

> 
> One way to fix that may be to have one ioflush thread for each userland
> filesystem. That way a broken filesystem will not prevent ioflush from
> working for others. But we have hit similar problems with others kernel
> threads.

sure, it can be a good idea.

YAMAMOTO Takashi

> 
> -- 
> Emmanuel Dreyfus
> http://hcpnet.free.fr/pubz
> m...@netbsd.org


Re: CVS commit: src/sys/net

2011-10-21 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: dyoung
> Date: Wed Oct 19 01:34:37 UTC 2011
> 
> Modified Files:
>   src/sys/net: if.c if.h
> 
> Log Message:
> Start to untangle the ifnet ioctls mess.
> 
> Add ifnet functions, if_mcast_op(), if_flags_set(), and if_addr_init()
> for adding/deleting multicast addresses, modifying the if_flags,
> and initializing local/remote addresses.  Make ifpromisc() use
> if_flags_set().  Protocols and network drivers should use these
> instead of ifp->if_ioctl() calls.  Subsequent commits will
> replace ifp->if_ioctl(SIOCADDMULTI| SIOCDELMULTI| SIOCSIFDSTADDR|
> SIOCINITIFADDR| SIOCSIFFLAGS) calls with calls to the new functions.
> 
> Use a mutex(9) to synchronize ifp->if_ioctl() calls originating in
> userland.  Also synchronize ifp->if_ioctl() calls with ifnet detachment
> and reclamation.

can you explain what you are trying to achieve with this percpu stuff?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.251 -r1.252 src/sys/net/if.c
> cvs rdiff -u -r1.151 -r1.152 src/sys/net/if.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libutil

2011-10-21 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Fri Oct 21 02:05:36 UTC 2011
> 
> Modified Files:
>   src/lib/libutil: Makefile
> 
> Log Message:
> Add proc_compare

you forgot to commit proc_compare.3?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.67 -r1.68 src/lib/libutil/Makefile
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/kern

2011-10-23 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: jym
> Date: Sun Oct 23 21:41:23 UTC 2011
> 
> Modified Files:
>   src/sys/kern: subr_workqueue.c
> 
> Log Message:
> Turn a workqueue(9) name into an array in the struct workqueue, rather
> than a const char *. This avoids keeping a reference to a string
> owned by caller (string could be allocated on stack).

what needs it?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.31 -r1.32 src/sys/kern/subr_workqueue.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/arch/m68k

2011-10-30 Thread YAMAMOTO Takashi
hi,

is a nointr pool safe for this pmap?
iirc x86 pmap allocates pv entries from kmem_map to avoid this kind of
recursion.

YAMAMOTO Takashi

> Module Name:  src
> Committed By: tsutsui
> Date: Sat Oct 29 18:26:20 UTC 2011
> 
> Modified Files:
>   src/sys/arch/m68k/include: pmap_motorola.h
>   src/sys/arch/m68k/m68k: pmap_motorola.c
> 
> Log Message:
> Use pool(9) for struct pv_entry allocations rather than
> uvm_km_alloc(9)/uvm_km_free(9) and ancient homegrown pv_page_info structures.
> 
> Calling uvm_km_free(9) during pmap_remove(9) could cause rw_lock error
> in uvm_unmap1() as noted in PR port-m68k/45519.
> 
> NetBSD/x68k (both real X68030 and XM6i emulator) still gets weird panic
> (corrupted kernel stack pointer?) on some heavy load:
> ---
> panic: MMU fault
> Stopped in pid 363.1 (X68k) at  netbsd:cpu_Debugger+0x6:  unlka6
> db> tr
> cpu_Debugger(4012004,8,1cbb528,2a618e0,2a5b000) + 6
> db>
> ---
> but it also occurs without this change so there might be some more bugs
> in m68k pmap...
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.33 -r1.34 src/sys/arch/m68k/include/pmap_motorola.h
> cvs rdiff -u -r1.62 -r1.63 src/sys/arch/m68k/m68k/pmap_motorola.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/arch/m68k

2011-10-31 Thread YAMAMOTO Takashi
hi,

>> is a nointr pool safe for this pmap?
> 
> I could be stupid in this area.
> 
>> iirc x86 pmap allocates pv entries from kmem_map to avoid this kind of
>> recursion.
> 
> I see, but x86 pmap has been changed to use pool_cache(9) with
> pool_allocator_meta during vmlocking merge.
> http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/arch/x86/x86/pmap.c.diff?r1=1.2&r2=1.3
> 
> Is it enough to use pool(9) with pool_allocator_meta (which uses kmem_map),
> or should I revert that pool change and just make uvm_km_alloc() in
> old pmap_alloc_pv() use kmem_map instead of kernel_map?

although i'm not familiar with this pmap, i think pool_allocator_meta is ok.

YAMAMOTO Takashi

> 
> ---
> Izumi Tsutsui


Re: CVS commit: src/lib/libterminfo

2011-11-02 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: roy
> Date: Wed Nov  2 12:09:26 UTC 2011
> 
> Modified Files:
>   src/lib/libterminfo: Makefile genhash genterms genthash
> Removed Files:
>   src/lib/libterminfo: compiled_terms.c hash.c termcap_hash.c
> 
> Log Message:
> Now that nbperf can generate the same file using the -p option,
> there is no longer a need to store these files in CVS.

i guess you need to tweak tools/tic?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.17 -r1.18 src/lib/libterminfo/Makefile
> cvs rdiff -u -r1.2 -r0 src/lib/libterminfo/compiled_terms.c
> cvs rdiff -u -r1.7 -r1.8 src/lib/libterminfo/genhash
> cvs rdiff -u -r1.3 -r1.4 src/lib/libterminfo/genterms
> cvs rdiff -u -r1.4 -r1.5 src/lib/libterminfo/genthash
> cvs rdiff -u -r1.4 -r0 src/lib/libterminfo/hash.c
> cvs rdiff -u -r1.3 -r0 src/lib/libterminfo/termcap_hash.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/net

2011-11-02 Thread YAMAMOTO Takashi
hi,

> On Thu, Oct 20, 2011 at 06:59:42AM +0000, YAMAMOTO Takashi wrote:
>> hi,
>> 
>> > Module Name:   src
>> > Committed By:  dyoung
>> > Date:  Wed Oct 19 01:34:37 UTC 2011
>> > 
>> > Modified Files:
>> >src/sys/net: if.c if.h
>> > 
>> > Log Message:
>> > Start to untangle the ifnet ioctls mess.
>> > 
>> > Add ifnet functions, if_mcast_op(), if_flags_set(), and if_addr_init()
>> > for adding/deleting multicast addresses, modifying the if_flags,
>> > and initializing local/remote addresses.  Make ifpromisc() use
>> > if_flags_set().  Protocols and network drivers should use these
>> > instead of ifp->if_ioctl() calls.  Subsequent commits will
>> > replace ifp->if_ioctl(SIOCADDMULTI| SIOCDELMULTI| SIOCSIFDSTADDR|
>> > SIOCINITIFADDR| SIOCSIFFLAGS) calls with calls to the new functions.
>> > 
>> > Use a mutex(9) to synchronize ifp->if_ioctl() calls originating in
>> > userland.  Also synchronize ifp->if_ioctl() calls with ifnet detachment
>> > and reclamation.
>> 
>> can you explain what you are trying to achieve with this percpu stuff?
> 
> I have tried to explain in some new comments.  Let me know if you still
> have questions.

which comments?
i don't understand how it's different from simply acquiring a kmutex_t.

YAMAMOTO Takashi

> 
> Dave
> 
> -- 
> David Young OJC Technologies is now Pixo
> dyo...@pixotech.com Urbana, IL   (217) 344-0444 x24


Re: CVS commit: src/sys/net

2011-11-06 Thread YAMAMOTO Takashi
hi,

isn't PR/40516 a problem for this particular usage?

YAMAMOTO Takashi

> Module Name:  src
> Committed By: dyoung
> Date: Wed Nov  2 01:17:59 UTC 2011
> 
> Modified Files:
>   src/sys/net: if_gre.c if_gre.h
> 
> Log Message:
> For simplicity's sake, use pcq(9) instead of my own circular-queue
> implementation.  Saves 45 lines of code.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.148 -r1.149 src/sys/net/if_gre.c
> cvs rdiff -u -r1.40 -r1.41 src/sys/net/if_gre.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/net

2011-11-07 Thread YAMAMOTO Takashi
> On Sun, Nov 06, 2011 at 11:43:06PM +0000, YAMAMOTO Takashi wrote:
>> hi,
>> 
>> isn't PR/40516 a problem for this particular usage?
> 
> Probably.  Should we use Andrew's version for now?

matt, can you take a look?

YAMAMOTO Takashi

> 
> Dave
> 
> -- 
> David Young OJC Technologies is now Pixo
> dyo...@pixotech.com Urbana, IL   (217) 344-0444 x24


Re: CVS commit: src/sys/nfs

2011-11-14 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: hannken
> Date: Sun Oct 30 12:00:28 UTC 2011
> 
> Modified Files:
>   src/sys/nfs: nfs_serv.c
> 
> Log Message:
> VOP_GETATTR() needs a shared lock at least.

this seems trying to lock the directory while holding its child locked.
ie. wrong locking order

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.160 -r1.161 src/sys/nfs/nfs_serv.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/arch/i386/i386

2011-11-20 Thread YAMAMOTO Takashi
hi,

> On Oct 31, 2011, at 1:42 PM, YAMAMOTO Takashi wrote:
> 
>> Module Name: src
>> Committed By:yamt
>> Date:Mon Oct 31 12:42:53 UTC 2011
>> 
>> Modified Files:
>>  src/sys/arch/i386/i386: dumpsys.c
>> 
>> Log Message:
>> dumpsys_seg: don't overwrite the previous mapping
> 
> With this change in place core dumps from ddb (reboot 104) no longer work
> on MP machines.
> 
> Before pmap_tlb_shootnow() always returned on the `tp->tp_count == 0' check.
> 
> Now it goes into the `remote' case and hangs hard trying to reach other CPUs.

thanks, i've reverted the changes for now.

YAMAMOTO Takashi

> 
> --
> Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)


Re: CVS commit: [yamt-pagecache] src/sys

2011-11-20 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: yamt
> Date: Sun Nov 20 10:52:35 UTC 2011
> 
> Modified Files:
>   src/sys/kern [yamt-pagecache]: init_main.c
>   src/sys/uvm [yamt-pagecache]: uvm.h uvm_extern.h uvm_init.c uvm_loan.c
>   uvm_meter.c uvm_page.c uvm_page.h uvm_page_status.c
> 
> Log Message:
> - fix page loaning  XXX make O->A loaning further

i meant: XXX make O->A loaning bitrot further

> - add some statistics
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.436.2.1 -r1.436.2.2 src/sys/kern/init_main.c
> cvs rdiff -u -r1.62.4.2 -r1.62.4.3 src/sys/uvm/uvm.h
> cvs rdiff -u -r1.176.2.3 -r1.176.2.4 src/sys/uvm/uvm_extern.h
> cvs rdiff -u -r1.41 -r1.41.4.1 src/sys/uvm/uvm_init.c
> cvs rdiff -u -r1.81.2.3 -r1.81.2.4 src/sys/uvm/uvm_loan.c
> cvs rdiff -u -r1.56.4.4 -r1.56.4.5 src/sys/uvm/uvm_meter.c
> cvs rdiff -u -r1.178.2.6 -r1.178.2.7 src/sys/uvm/uvm_page.c
> cvs rdiff -u -r1.73.2.6 -r1.73.2.7 src/sys/uvm/uvm_page.h
> cvs rdiff -u -r1.1.2.4 -r1.1.2.5 src/sys/uvm/uvm_page_status.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.



Re: CVS commit: src/sys/fs/union

2011-11-21 Thread YAMAMOTO Takashi
hi,

do you have any plan on fs/unionfs?  eg. remove it

YAMAMOTO Takashi

> Module Name:  src
> Committed By: hannken
> Date: Mon Nov 21 18:29:23 UTC 2011
> 
> Modified Files:
>   src/sys/fs/union: union.h union_subr.c union_vfsops.c union_vnops.c
> 
> Log Message:
> Replace flag based union node locking with generic vnode lock, support
> shared and nowait locks and protect un_uppervp and un_*sz with mutex.
> 
> Mark file system MPSAFE.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.21 -r1.22 src/sys/fs/union/union.h
> cvs rdiff -u -r1.52 -r1.53 src/sys/fs/union/union_subr.c
> cvs rdiff -u -r1.64 -r1.65 src/sys/fs/union/union_vfsops.c
> cvs rdiff -u -r1.48 -r1.49 src/sys/fs/union/union_vnops.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys

2011-12-20 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Tue Dec 20 23:56:29 UTC 2011
> 
> Modified Files:
>   src/sys/compat/linux/common: linux_socket.c
>   src/sys/dev: kttcp.c
>   src/sys/kern: sys_socket.c uipc_socket.c uipc_socket2.c uipc_syscalls.c
>   src/sys/miscfs/fifofs: fifo_vnops.c
>   src/sys/netiso: tp_usrreq.c
>   src/sys/sys: socket.h socketvar.h
> 
> Log Message:
> - Eliminate so_nbio and turn it into a bit SS_NBIO in so_state.

why?
i thought the reason of having this as a separate member was performance.

YAMAMOTO Takashi

> - Introduce MSG_NBIO so that we can turn non blocking i/o on a per call basis
> - Use MSG_NBIO to fix the XXX: multi-threaded issues on the fifo sockets.
> - Don't set SO_CANTRCVMORE, if we were interrupted (perhaps do it for all
>   errors?).
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.110 -r1.111 src/sys/compat/linux/common/linux_socket.c
> cvs rdiff -u -r1.28 -r1.29 src/sys/dev/kttcp.c
> cvs rdiff -u -r1.64 -r1.65 src/sys/kern/sys_socket.c
> cvs rdiff -u -r1.205 -r1.206 src/sys/kern/uipc_socket.c
> cvs rdiff -u -r1.109 -r1.110 src/sys/kern/uipc_socket2.c
> cvs rdiff -u -r1.148 -r1.149 src/sys/kern/uipc_syscalls.c
> cvs rdiff -u -r1.70 -r1.71 src/sys/miscfs/fifofs/fifo_vnops.c
> cvs rdiff -u -r1.40 -r1.41 src/sys/netiso/tp_usrreq.c
> cvs rdiff -u -r1.100 -r1.101 src/sys/sys/socket.h
> cvs rdiff -u -r1.126 -r1.127 src/sys/sys/socketvar.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys

2011-12-28 Thread YAMAMOTO Takashi
hi,

> On Dec 21,  5:07am, y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) wrote:
> -- Subject: Re: CVS commit: src/sys
> 
> | why?
> | i thought the reason of having this as a separate member was performance.
> 
> All the code paths that use NBIO check bits in so_state... I would like to
> see the benchmark that proves this.

it's about the fcntl side, for which you added more locking.

YAMAMOTO Takashi

> 
> christos


Re: CVS commit: src

2012-01-24 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Wed Jan 25 00:28:36 UTC 2012
> 
> Modified Files:
>   src/lib/libc/sys: dup.2 fcntl.2 getsockopt.2 kqueue.2 open.2 pipe.2
>   socket.2
>   src/sys/kern: kern_descrip.c kern_event.c sys_descrip.c sys_generic.c
>   sys_pipe.c uipc_socket.c uipc_syscalls.c
>   src/sys/sys: fcntl.h filedesc.h socket.h
> 
> Log Message:
> As discussed in tech-kern, provide the means to prevent delivery of SIGPIPE
> on EPIPE for all file descriptor types:
> 
> - provide O_NOSIGPIPE for open,kqueue1,pipe2,dup3,fcntl(F_{G,S}ETFL) [NetBSD]
> - provide SOCK_NOSIGPIPE for socket,socketpair [NetBSD]
> - provide SO_NOSIGPIPE for {g,s}seckopt [NetBSD/FreeBSD/MacOSX]
> - provide F_{G,S}ETNOSIGPIPE for fcntl [MacOSX]

please use proper locking for f_flag and so_options.

i don't think these redundant compat api are worth the complexity.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.27 -r1.28 src/lib/libc/sys/dup.2 src/lib/libc/sys/pipe.2
> cvs rdiff -u -r1.39 -r1.40 src/lib/libc/sys/fcntl.2
> cvs rdiff -u -r1.34 -r1.35 src/lib/libc/sys/getsockopt.2
> cvs rdiff -u -r1.31 -r1.32 src/lib/libc/sys/kqueue.2
> cvs rdiff -u -r1.50 -r1.51 src/lib/libc/sys/open.2
> cvs rdiff -u -r1.37 -r1.38 src/lib/libc/sys/socket.2
> cvs rdiff -u -r1.217 -r1.218 src/sys/kern/kern_descrip.c
> cvs rdiff -u -r1.74 -r1.75 src/sys/kern/kern_event.c
> cvs rdiff -u -r1.23 -r1.24 src/sys/kern/sys_descrip.c
> cvs rdiff -u -r1.127 -r1.128 src/sys/kern/sys_generic.c
> cvs rdiff -u -r1.134 -r1.135 src/sys/kern/sys_pipe.c
> cvs rdiff -u -r1.206 -r1.207 src/sys/kern/uipc_socket.c
> cvs rdiff -u -r1.150 -r1.151 src/sys/kern/uipc_syscalls.c
> cvs rdiff -u -r1.41 -r1.42 src/sys/sys/fcntl.h
> cvs rdiff -u -r1.61 -r1.62 src/sys/sys/filedesc.h
> cvs rdiff -u -r1.104 -r1.105 src/sys/sys/socket.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src

2012-01-30 Thread YAMAMOTO Takashi
hi,

is the change in conf/std still necessary?

YAMAMOTO Takashi

> Module Name:  src
> Committed By: tls
> Date: Sat Dec 17 20:05:40 UTC 2011
> 
> Modified Files:
>   src/share/man/man4: rnd.4
>   src/share/man/man9: cprng.9 rnd.9
>   src/sys/conf: files
>   src/sys/crypto/nist_ctr_drbg: nist_ctr_drbg_aes128.h
>   nist_ctr_drbg_aes256.h
>   src/sys/dev: rnd.c rndpool.c
>   src/sys/dev/iscsi: iscsi_text.c
>   src/sys/dist/pf/netinet: tcp_rndiss.c
>   src/sys/kern: init_sysctl.c subr_cprng.c
>   src/sys/net: if_spppsubr.c
>   src/sys/netinet: tcp_subr.c
>   src/sys/rump/dev/lib/librnd: Makefile
>   src/sys/rump/librump/rumpkern: cprng_stub.c
>   src/sys/sys: cprng.h param.h rnd.h
> Added Files:
>   src/sys/dev: rndpseudo.c
> 
> Log Message:
> Separate /dev/random pseudodevice implemenation from kernel entropy pool
> implementation.  Rewrite pseudodevice code to use cprng_strong(9).
> 
> The new pseudodevice is cloning, so each caller gets bits from a stream
> generated with its own key.  Users of /dev/urandom get their generators
> keyed on a "best effort" basis -- the kernel will rekey generators
> whenever the entropy pool hits the high water mark -- while users of
> /dev/random get their generators rekeyed every time key-length bits
> are output.
> 
> The underlying cprng_strong API can use AES-256 or AES-128, but we use
> AES-128 because of concerns about related-key attacks on AES-256.  This
> improves performance (and reduces entropy pool depletion) significantly
> for users of /dev/urandom but does cause users of /dev/random to rekey
> twice as often.
> 
> Also fixes various bugs (including some missing locking and a reseed-counter
> overflow in the CTR_DRBG code) found while testing this.
> 
> For long reads, this generator is approximately 20 times as fast as the
> old generator (dd with bs=64K yields 53MB/sec on 2Ghz Core2 instead of
> 2.5MB/sec) and also uses a separate mutex per instance so concurrency
> is greatly improved.  For reads of typical key sizes for modern
> cryptosystems (16-32 bytes) performance is about the same as the old
> code: a little better for 32 bytes, a little worse for 16 bytes.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.16 -r1.17 src/share/man/man4/rnd.4
> cvs rdiff -u -r1.3 -r1.4 src/share/man/man9/cprng.9
> cvs rdiff -u -r1.18 -r1.19 src/share/man/man9/rnd.9
> cvs rdiff -u -r1.1032 -r1.1033 src/sys/conf/files
> cvs rdiff -u -r1.1 -r1.2 src/sys/crypto/nist_ctr_drbg/nist_ctr_drbg_aes128.h \
> src/sys/crypto/nist_ctr_drbg/nist_ctr_drbg_aes256.h
> cvs rdiff -u -r1.88 -r1.89 src/sys/dev/rnd.c
> cvs rdiff -u -r1.21 -r1.22 src/sys/dev/rndpool.c
> cvs rdiff -u -r0 -r1.1 src/sys/dev/rndpseudo.c
> cvs rdiff -u -r1.2 -r1.3 src/sys/dev/iscsi/iscsi_text.c
> cvs rdiff -u -r1.3 -r1.4 src/sys/dist/pf/netinet/tcp_rndiss.c
> cvs rdiff -u -r1.185 -r1.186 src/sys/kern/init_sysctl.c
> cvs rdiff -u -r1.4 -r1.5 src/sys/kern/subr_cprng.c
> cvs rdiff -u -r1.124 -r1.125 src/sys/net/if_spppsubr.c
> cvs rdiff -u -r1.243 -r1.244 src/sys/netinet/tcp_subr.c
> cvs rdiff -u -r1.2 -r1.3 src/sys/rump/dev/lib/librnd/Makefile
> cvs rdiff -u -r1.3 -r1.4 src/sys/rump/librump/rumpkern/cprng_stub.c
> cvs rdiff -u -r1.3 -r1.4 src/sys/sys/cprng.h
> cvs rdiff -u -r1.397 -r1.398 src/sys/sys/param.h
> cvs rdiff -u -r1.27 -r1.28 src/sys/sys/rnd.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/kern

2012-01-30 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: rmind
> Date: Mon Jan 30 21:05:40 UTC 2012
> 
> Modified Files:
>   src/sys/kern: subr_kmem.c
> 
> Log Message:
> Fix for KMEM_GUARD; do not use it from interrupt context.

kmem_zalloc still seems broken and anyway the test looks too fragile.
how about simply moving the #ifdef blocks to callers?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.40 -r1.41 src/sys/kern/subr_kmem.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/kern

2012-02-15 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: martin
> Date: Wed Feb 15 11:59:30 UTC 2012
> 
> Modified Files:
>   src/sys/kern: kern_exit.c
> 
> Log Message:
> Fix fallout from the new tests exercising all error paths: do not deactivate
> the pmap of a vmspace-less child of a posix spawn operation that never
> made it to userland.

please take a look at uvm_proc_exit().  it borrows proc0.p_vmspace
for exiting process.

while which of these methods is cleaner is probably a matter of taste,
these two cases should use the same method.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.235 -r1.236 src/sys/kern/kern_exit.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libc

2012-03-01 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: dholland
> Date: Fri Feb 24 16:06:39 UTC 2012
> 
> Modified Files:
>   src/lib/libc: shlib_version
> 
> Log Message:
> Note that gets() is finally dead in C11 and can be removed if we ever
> bump libc.

it doesn't sound good or realistic to me.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.228 -r1.229 src/lib/libc/shlib_version
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/sys

2012-03-18 Thread YAMAMOTO Takashi
hi,

> In article <20120318191308.ga10...@britannica.bec.de>,
> Joerg Sonnenberger   wrote:
>>On Sat, Mar 17, 2012 at 05:30:31PM -0400, Christos Zoulas wrote:
>>> Module Name:src
>>> Committed By:   christos
>>> Date:   Sat Mar 17 21:30:30 UTC 2012
>>> 
>>> Modified Files:
>>> src/sys/sys: types.h
>>> 
>>> Log Message:
>>> PR/44847: Jukka Ruohonen: blksize_t should be signed.
>>> http://pubs.opengroup.org/onlinepubs/95399/basedefs/sys/types.h.html
>>
>>I dislike the change. What is the justification for requiring this to be
>>signed? There are good reasons for having it be unsigned, e.g. getting
>>more efficient code by default.
> 
> va_blocksize is signed (long) and most of the 360+ userland occurances
> assume it is signed.

where did you count the number?
the occurrences in our tree doesn't seem so many.

YAMAMOTO Takashi

> 
> christos


Re: CVS commit: [matt-nb5-mips64] src/sys/uvm

2012-04-11 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: matt
> Date: Thu Apr 12 01:40:27 UTC 2012
> 
> Modified Files:
>   src/sys/uvm [matt-nb5-mips64]: uvm_extern.h uvm_fault.c uvm_km.c
>   uvm_meter.c uvm_pdaemon.c uvm_pdaemon.h uvm_pdpolicy.h
>   uvm_pdpolicy_clock.c uvm_stat.c
> 
> Log Message:
> Separate object-less anon pages out of the active list if there is no swap
> device.  Make uvm_reclaimable and uvm.*estimatable understand colors and
> kmem allocations.

i like the idea.

- why the queue is inside the pdpolicy?

- why don't you use PQ_SWAPBACKED?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.148.4.2.4.5 -r1.148.4.2.4.6 src/sys/uvm/uvm_extern.h
> cvs rdiff -u -r1.125.6.1.4.4 -r1.125.6.1.4.5 src/sys/uvm/uvm_fault.c
> cvs rdiff -u -r1.101.4.2.4.9 -r1.101.4.2.4.10 src/sys/uvm/uvm_km.c
> cvs rdiff -u -r1.49.16.3 -r1.49.16.4 src/sys/uvm/uvm_meter.c
> cvs rdiff -u -r1.93.4.2.4.8 -r1.93.4.2.4.9 src/sys/uvm/uvm_pdaemon.c
> cvs rdiff -u -r1.15 -r1.15.28.1 src/sys/uvm/uvm_pdaemon.h
> cvs rdiff -u -r1.3.62.1 -r1.3.62.2 src/sys/uvm/uvm_pdpolicy.h
> cvs rdiff -u -r1.12.16.4 -r1.12.16.5 src/sys/uvm/uvm_pdpolicy_clock.c
> cvs rdiff -u -r1.31.12.3 -r1.31.12.4 src/sys/uvm/uvm_stat.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: [matt-nb5-mips64] src/sys/uvm

2012-04-11 Thread YAMAMOTO Takashi
> 
> On Apr 11, 2012, at 7:34 PM, YAMAMOTO Takashi wrote:
> 
>> hi,
>> 
>>> Module Name:src
>>> Committed By:   matt
>>> Date:   Thu Apr 12 01:40:27 UTC 2012
>>> 
>>> Modified Files:
>>> src/sys/uvm [matt-nb5-mips64]: uvm_extern.h uvm_fault.c uvm_km.c
>>> uvm_meter.c uvm_pdaemon.c uvm_pdaemon.h uvm_pdpolicy.h
>>> uvm_pdpolicy_clock.c uvm_stat.c
>>> 
>>> Log Message:
>>> Separate object-less anon pages out of the active list if there is no swap
>>> device.  Make uvm_reclaimable and uvm.*estimatable understand colors and
>>> kmem allocations.
>> 
>> i like the idea.
>> 
>> - why the queue is inside the pdpolicy?
>> 
>> - why don't you use PQ_SWAPBACKED?
> 
> that's only set when a swap slot has been allocated for it.  Since there's no 
> swap, it'll never be set.

the availability of swap slots doesn't affect PQ_SWAPBACKED.
it's always set for anon/aobj pages.

YAMAMOTO Takashi

> 
> The problem I'm attacking is memory exhaustion.  Making sure that's pdaemon 
> isn't consuming too many resources when it can't do anything.  the pdaemon 
> would spin continuously with dirty reactivations and the above was an attempt 
> to stop that.
> 
> It's still a work-in-progress.


Re: CVS commit: src

2012-04-22 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Fri Apr 20 17:31:30 UTC 2012
> 
> Modified Files:
>   src/include: stdlib.h
>   src/lib/libc/compat/include: stdlib.h
>   src/lib/libc/compat/stdlib: Makefile.inc
> Added Files:
>   src/lib/libc/compat/stdlib: compat_putenv.c
> 
> Log Message:
> PR/46360: YAMAMOTO Takashi: Restore NetBSD-5 compatibility with putenv()
> copying the passed string (which is not ToG compliant), instead of using
> it directly in the environment arrat as it should. Needs to be pulled up
> to NetBSd-6.

thanks for a quick fix.
but why the compat code is different from the netbsd-5 code?
this version leaks memory if called repeatedly, doesn't it?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.97 -r1.98 src/include/stdlib.h
> cvs rdiff -u -r1.4 -r1.5 src/lib/libc/compat/include/stdlib.h
> cvs rdiff -u -r1.2 -r1.3 src/lib/libc/compat/stdlib/Makefile.inc
> cvs rdiff -u -r0 -r1.1 src/lib/libc/compat/stdlib/compat_putenv.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/arch

2012-05-15 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: joerg
> Date: Mon May  7 16:16:44 UTC 2012
> 
> Modified Files:
>   src/sys/arch/amd64/include: vmparam.h
>   src/sys/arch/i386/include: vmparam.h
> 
> Log Message:
> Raise per-image text size limit to 256MB. 64MB has seen already, so
> provide some margin of grows.

just curious; what program was it?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.29 -r1.30 src/sys/arch/amd64/include/vmparam.h
> cvs rdiff -u -r1.73 -r1.74 src/sys/arch/i386/include/vmparam.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src

2012-07-03 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: reinoud
> Date: Sat Jun 30 15:03:58 UTC 2012
> 
> Modified Files:
>   src/distrib/sets/lists/modules: md.amd64 md.i386
>   src/etc/etc.amd64: Makefile.inc
>   src/etc/etc.i386: Makefile.inc
>   src/sys/arch/usermode/conf: Makefile.usermode
>   src/sys/modules: Makefile
> Added Files:
>   src/sys/arch/amd64/conf: GENERIC_USERMODE
>   src/sys/arch/i386/conf: GENERIC_USERMODE
> Removed Files:
>   src/sys/arch/usermode/conf: GENERIC.amd64 GENERIC.i386
> 
> Log Message:
> Move i386 and amd64 usermode configurations to their respective directories
> and make the usermode kernels buildalbe under build.sh.
> 
> The resulting kernels are build and packaged correctly as are the associated
> modules.

this seems to break cross build.
(target amd64, host i386)

YAMAMOTO Takashi

cc1: warnings being treated as errors
/siro/nbsd/src/sys/arch/usermode/usermode/thunk.c: In function 
'thunk_syscallemu_init':
/siro/nbsd/src/sys/arch/usermode/usermode/thunk.c:118:34: error: cast from 
pointer to integer of different size
/siro/nbsd/src/sys/arch/usermode/usermode/thunk.c:118:53: error: cast from 
pointer to integer of different size

*** Failed target:  thunk.o

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.32 -r1.33 src/distrib/sets/lists/modules/md.amd64
> cvs rdiff -u -r1.38 -r1.39 src/distrib/sets/lists/modules/md.i386
> cvs rdiff -u -r1.12 -r1.13 src/etc/etc.amd64/Makefile.inc
> cvs rdiff -u -r1.65 -r1.66 src/etc/etc.i386/Makefile.inc
> cvs rdiff -u -r0 -r1.1 src/sys/arch/amd64/conf/GENERIC_USERMODE
> cvs rdiff -u -r0 -r1.1 src/sys/arch/i386/conf/GENERIC_USERMODE
> cvs rdiff -u -r1.3 -r0 src/sys/arch/usermode/conf/GENERIC.amd64
> cvs rdiff -u -r1.5 -r0 src/sys/arch/usermode/conf/GENERIC.i386
> cvs rdiff -u -r1.31 -r1.32 src/sys/arch/usermode/conf/Makefile.usermode
> cvs rdiff -u -r1.106 -r1.107 src/sys/modules/Makefile
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/usr.sbin/nfsd

2012-08-13 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Mon Aug 13 08:19:11 UTC 2012
> 
> Modified Files:
>   src/usr.sbin/nfsd: nfsd.8 nfsd.c
> 
> Log Message:
> Let nfsd behave like all other programs: tries to use both inet4 and inet6
> by default and both udp and tcp: -4 uses only inet4, -6 uses only inet6,
> -t uses only tcp, -u uses only udp. For compatibility, we detect old option
> usage, we warn, and DTRT.

i'm not sure if it's a good direction to go because "uses only XXX"
style options are not scalable if there can be more than two choices
of XXX.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.21 -r1.22 src/usr.sbin/nfsd/nfsd.8
> cvs rdiff -u -r1.58 -r1.59 src/usr.sbin/nfsd/nfsd.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libpthread

2012-11-21 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Wed Nov 21 19:19:25 UTC 2012
> 
> Modified Files:
>   src/lib/libpthread: pthread_int.h pthread_specific.c pthread_tsd.c
> 
> Log Message:
> Replace the simple implementation of pthread_key_{create,destroy}
> and pthread_{g,s}etspecific functions, to one that invalidates
> values of keys in other threads when pthread_key_delete() is called.
> This fixes chromium, which expects pthread_key_delete() to do
> cleanup in all threads.

i think pthread_key_delete should not call dtor.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.87 -r1.88 src/lib/libpthread/pthread_int.h
> cvs rdiff -u -r1.23 -r1.24 src/lib/libpthread/pthread_specific.c
> cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_tsd.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys

2012-11-30 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: christos
> Date: Thu Nov 29 02:07:21 UTC 2012
> 
> Modified Files:
>   src/sys/netinet: ip_input.c portalgo.c portalgo.h
>   src/sys/netinet6: ip6_input.c
> 
> Log Message:
> Add a new sysctl to mark ports as reserved, so that they are not used in
> the anonymous or reserved port allocation.

what's the purpose of this feature?

fd_set doesn't seem like an appropriate structure for this.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.302 -r1.303 src/sys/netinet/ip_input.c
> cvs rdiff -u -r1.1 -r1.2 src/sys/netinet/portalgo.c \
> src/sys/netinet/portalgo.h
> cvs rdiff -u -r1.140 -r1.141 src/sys/netinet6/ip6_input.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/sys

2013-03-10 Thread YAMAMOTO Takashi
hi,

openvswitch has the following CTASSERT equivalent, which doesn't
require __COUNTER__.

#define OFP_ASSERT(EXPR)\
extern int (*build_assert(void))[ sizeof(struct {   \
unsigned int build_assert_failed : (EXPR) ? 1 : -1; })]

any comments from C lawyers?

YAMAMOTO Takashi

> Module Name:  src
> Committed By: pooka
> Date: Wed Mar  6 18:16:58 UTC 2013
> 
> Modified Files:
>   src/sys/sys: ucontext.h
> 
> Log Message:
> Move CTASSERT a few lines down so as to not collide with the CTASSERT
> in rtsock.c (relevant for compilers without __COUNTER__).  Yes,
> it's a really cheap tweak.  Tweak better if it tickles your tweakybone.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.17 -r1.18 src/sys/sys/ucontext.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 
> 


Re: CVS commit: src/sys/kern

2013-03-11 Thread YAMAMOTO Takashi
hi,

> Module Name:  src
> Committed By: pooka
> Date: Mon Mar 11 21:37:54 UTC 2013
> 
> Modified Files:
>   src/sys/kern: subr_pool.c
> 
> Log Message:
> In pool_cache_put_slow(), pool_get() can block (it does mutex_enter()),
> so we need to retry if curlwp took a context switch during the call.
> Otherwise, CPU-local invariants can get screwed up:
> 
> panic: kernel diagnostic assertion "cur->pcg_avail == cur->pcg_size" 
> failed
> 
> This is (was) very easy to reproduce by just running:
> 
>   while : ; do RUMP_NCPU=32 ./a.out ; done
> 
> where a.out only calls rump_init().  But, any situation there's contention
> and a pool doesn't have emptygroups would do.

depending on mutex_init's arguments (type and ipl), a mutex can be
spin or adaptive.
rump mutex implementation should honor the behaviour, i guess.

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.199 -r1.200 src/sys/kern/subr_pool.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libpthread

2013-03-28 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: christos
> Date: Thu Mar 28 18:07:12 UTC 2013
> 
> Modified Files:
>   src/lib/libpthread: pthread_cond.c
> 
> Log Message:
> PR/47703: Yasushi Oshima: pthread_cond_timedwait() does not wait
> after call pthread_condattr_setclock(CLOCK_MONOTONIC)
> 
> _lwp_park(2) expects a realtime clock, and it gets passed a monotonic
> one.  Since monotonic < real, it never sleeps. This patch adjusts
> the monotonic clock to be a real one before it passes is to
> _lwp_park(2). This is the minimal hacky fix and it will be fixed
> properly in _lwp_park(2) in the future.
> 
> XXX: pullup to 6.

"mono" escaping its scope?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.59 -r1.60 src/lib/libpthread/pthread_cond.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libpthread

2013-03-31 Thread YAMAMOTO Takashi
> In article <20130329032804.23b1514a...@mail.netbsd.org>,
> YAMAMOTO Takashi  wrote:
>>
>>"mono" escaping its scope?
> 
> Yes, I already added it to lwp_park to avoid the hack but not using
> the syscall yet until we decide what we need to pass to lwp_park
> to avoid priority inversion and fix the races in pthread_cond_*.
> 
> christos

i meant the following code.
struct timespec mono will be used after the end of the block.
is it safe?

if (pthread_cond_getclock(cond) == CLOCK_MONOTONIC) {
struct timespec mono, real;
if (clock_gettime(CLOCK_REALTIME, &real) == -1 ||
clock_gettime(CLOCK_MONOTONIC, &mono) == -1)
return errno;
timespecsub(abstime, &mono, &mono);
timespecadd(&mono, &real, &mono);
abstime = &mono;
}



Re: CVS commit: src/lib/libc/locale

2013-04-15 Thread YAMAMOTO Takashi
hi,

> On Mon, Apr 15, 2013 at 11:40:57PM +0900, Takehiko NOZAKI wrote:
>> POSIX2008 spec say, *_l func with invalid locale handle may EINVAL.
>> NULL or (locale_t)0 is invalid locale handle.
>> why are you think fallback to C locale?
> 
> That's what Apple and FreeBSD provide and which is actually quite
> useful. Wanting access to the C locale is the most common case for many
> locale issues. Making that easy sounds like a good idea in general.

is there consumers of this "extension" in the field?

an application can have a global c_locale = newlocale("C")
if it wants an easy access to the C locale.

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src/lib/libc/locale

2013-04-16 Thread YAMAMOTO Takashi
> On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
>> so that the struct _locale __C_locale in libc is much more wasteful.
> 
> I should add that it is an internal detail and the way the composed C
> locale is stored can and likely will change later. So the way it is
> essentially a copy of (old) global locale is just a way to be minimally
> intrusive.
> 
> Joerg

i care the API.

if you really want it be in libc, how about having libc provide a
"locale_t get_static_c_locale();" style API rather than using NULL?
it's better because 1) less code in *_l, 2) autoconf-like can detect
the extension easily, and 3) a portable application can trivially
have a fallback implementation using newlocale+pthread_once.

YAMAMOTO Takashi


Re: CVS commit: src

2013-04-16 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: joerg
> Date: Tue Apr 16 21:44:08 UTC 2013
> 
> Modified Files:
>   src/common/lib/libc/stdlib: _strtol.h _strtoul.h strtoll.c strtoull.c
>   strtoumax.c
>   src/include: inttypes.h stdlib.h
>   src/lib/libc: shlib_version
>   src/lib/libc/citrus: citrus_bcs_strtol.c citrus_bcs_strtoul.c
>   src/lib/libc/include: namespace.h
>   src/lib/libc/stdlib: strtoimax.c
> 
> Log Message:
> Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
> strtol_l unless in tools mode. Add note to retire the BCS code on the
> next libc major bump.

is it safe to make tools use host's strtoul?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.4 -r1.5 src/common/lib/libc/stdlib/_strtol.h \
> src/common/lib/libc/stdlib/_strtoul.h
> cvs rdiff -u -r1.6 -r1.7 src/common/lib/libc/stdlib/strtoll.c
> cvs rdiff -u -r1.5 -r1.6 src/common/lib/libc/stdlib/strtoull.c \
> src/common/lib/libc/stdlib/strtoumax.c
> cvs rdiff -u -r1.8 -r1.9 src/include/inttypes.h
> cvs rdiff -u -r1.100 -r1.101 src/include/stdlib.h
> cvs rdiff -u -r1.239 -r1.240 src/lib/libc/shlib_version
> cvs rdiff -u -r1.2 -r1.3 src/lib/libc/citrus/citrus_bcs_strtol.c
> cvs rdiff -u -r1.3 -r1.4 src/lib/libc/citrus/citrus_bcs_strtoul.c
> cvs rdiff -u -r1.157 -r1.158 src/lib/libc/include/namespace.h
> cvs rdiff -u -r1.7 -r1.8 src/lib/libc/stdlib/strtoimax.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libc/locale

2013-04-17 Thread YAMAMOTO Takashi
> On Tue, Apr 16, 2013 at 11:34:52PM +0000, YAMAMOTO Takashi wrote:
>> > On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
>> >> so that the struct _locale __C_locale in libc is much more wasteful.
>> > 
>> > I should add that it is an internal detail and the way the composed C
>> > locale is stored can and likely will change later. So the way it is
>> > essentially a copy of (old) global locale is just a way to be minimally
>> > intrusive.
>> > 
>> > Joerg
>> 
>> i care the API.
>> 
>> if you really want it be in libc, how about having libc provide a
>> "locale_t get_static_c_locale();" style API rather than using NULL?
>> it's better because 1) less code in *_l, 2) autoconf-like can detect
>> the extension easily, and 3) a portable application can trivially
>> have a fallback implementation using newlocale+pthread_once.
> 
> It adds more cost on the caller side. So far, there are three mechanisms
> available (especially for libraries):
> 
> (1) Adhoc access to internal locale state. This is used with glibc.
> (2) Explicit newlocale().
> (3) Implicit access via 0 argument to *_l.
> 
> The first one is clearly a hack and not acceptable. Portable code can
> always conditionally use (2), but it requires additional setup and
> storage cost. (3) is used by Apple (which is where a large part of the
> *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
> certainly easier to use. Exposing it via a special library call is also
> possible and effectively a way to implement (2) by a static wrapper.
> It still adds more cost to every caller and this is a classic case where
> there are typically much more callers.
> 
> Joerg

(3) adds small costs to every calls of *_l, even when the extension
is not used.  i sounds worse than a one-time cost of (2) to me.

YAMAMOTO Takashi


Re: CVS commit: src/lib/libc/locale

2013-04-21 Thread YAMAMOTO Takashi
> On Thu, Apr 18, 2013 at 06:58:42AM +0000, YAMAMOTO Takashi wrote:
>> > On Tue, Apr 16, 2013 at 11:34:52PM +0000, YAMAMOTO Takashi wrote:
>> >> > On Tue, Apr 16, 2013 at 11:39:50PM +0900, Takehiko NOZAKI wrote:
>> >> >> so that the struct _locale __C_locale in libc is much more wasteful.
>> >> > 
>> >> > I should add that it is an internal detail and the way the composed C
>> >> > locale is stored can and likely will change later. So the way it is
>> >> > essentially a copy of (old) global locale is just a way to be minimally
>> >> > intrusive.
>> >> > 
>> >> > Joerg
>> >> 
>> >> i care the API.
>> >> 
>> >> if you really want it be in libc, how about having libc provide a
>> >> "locale_t get_static_c_locale();" style API rather than using NULL?
>> >> it's better because 1) less code in *_l, 2) autoconf-like can detect
>> >> the extension easily, and 3) a portable application can trivially
>> >> have a fallback implementation using newlocale+pthread_once.
>> > 
>> > It adds more cost on the caller side. So far, there are three mechanisms
>> > available (especially for libraries):
>> > 
>> > (1) Adhoc access to internal locale state. This is used with glibc.
>> > (2) Explicit newlocale().
>> > (3) Implicit access via 0 argument to *_l.
>> > 
>> > The first one is clearly a hack and not acceptable. Portable code can
>> > always conditionally use (2), but it requires additional setup and
>> > storage cost. (3) is used by Apple (which is where a large part of the
>> > *_l interface originates from) and FreeBSD. It is orthogonal to (2) and
>> > certainly easier to use. Exposing it via a special library call is also
>> > possible and effectively a way to implement (2) by a static wrapper.
>> > It still adds more cost to every caller and this is a classic case where
>> > there are typically much more callers.
>> > 
>> > Joerg
>> 
>> (3) adds small costs to every calls of *_l, even when the extension
>> is not used.  i sounds worse than a one-time cost of (2) to me.
> 
> (2) still needs to load the address (instead of a constant), so it isn't
> free either. Given that this is very popular as functionality and at
> least on modern CPUs often implementable as conditional-move, it sounds
> like a much better trade off.

very popular?  i'm not aware of a single software which uses this extension.
can you provide examples?

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src

2013-04-23 Thread YAMAMOTO Takashi
> On Wed, Apr 17, 2013 at 06:32:30AM +0000, YAMAMOTO Takashi wrote:
>> > Module Name:   src
>> > Committed By:  joerg
>> > Date:  Tue Apr 16 21:44:08 UTC 2013
>> > 
>> > Modified Files:
>> >src/common/lib/libc/stdlib: _strtol.h _strtoul.h strtoll.c strtoull.c
>> >strtoumax.c
>> >src/include: inttypes.h stdlib.h
>> >src/lib/libc: shlib_version
>> >src/lib/libc/citrus: citrus_bcs_strtol.c citrus_bcs_strtoul.c
>> >src/lib/libc/include: namespace.h
>> >src/lib/libc/stdlib: strtoimax.c
>> > 
>> > Log Message:
>> > Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
>> > strtol_l unless in tools mode. Add note to retire the BCS code on the
>> > next libc major bump.
>> 
>> is it safe to make tools use host's strtoul?
> 
> I think so. Tools shouldn't be using locales.

as you know, use C locale != stop using locale.
i don't know if it can be an actual problem for netbsd as we likely
have other code which assumes PCS compatibility between host and netbsd,
though.  on the other hand, this code is intended to be portable than
the rest of libc because there's a plan to create a portable version of
citrus iconv library.
you'd be better to ask the author (tnozaki) to clarify the intention of
the code before simplifying it.

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src/sys/arch

2013-06-24 Thread YAMAMOTO Takashi
hi,

why do you need to make *.S include another *.S
rather than tweaking makefile?

YAMAMOTO Takashi

> Module Name:  src
> Committed By: uebayasi
> Date: Tue Jun 25 00:27:22 UTC 2013
> 
> Modified Files:
>   src/sys/arch/amd64/amd64: vector.S
>   src/sys/arch/i386/i386: vector.S
> Added Files:
>   src/sys/arch/amd64/amd64: amd64_trap.S
>   src/sys/arch/i386/i386: i386_trap.S i386_trap_ipkdb.S
> 
> Log Message:
> Split these to improve diffability.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r0 -r1.1 src/sys/arch/amd64/amd64/amd64_trap.S
> cvs rdiff -u -r1.43 -r1.44 src/sys/arch/amd64/amd64/vector.S
> cvs rdiff -u -r0 -r1.1 src/sys/arch/i386/i386/i386_trap.S \
> src/sys/arch/i386/i386/i386_trap_ipkdb.S
> cvs rdiff -u -r1.61 -r1.62 src/sys/arch/i386/i386/vector.S
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/sys/arch

2013-06-24 Thread YAMAMOTO Takashi
hi,

> Because I don't want to change resulting binaries (*.o) for now.  I
> think I'll "switch" these when cleaned up and merged and moved to
> sys/arch/x86.  How do you think?

fine for me as far as this inclusion is temporary.

YAMAMOTO Takashi

> 
> On Tue, Jun 25, 2013 at 12:17 PM, YAMAMOTO Takashi
>  wrote:
>> hi,
>>
>> why do you need to make *.S include another *.S
>> rather than tweaking makefile?
>>
>> YAMAMOTO Takashi
>>
>>> Module Name:  src
>>> Committed By: uebayasi
>>> Date: Tue Jun 25 00:27:22 UTC 2013
>>>
>>> Modified Files:
>>>   src/sys/arch/amd64/amd64: vector.S
>>>   src/sys/arch/i386/i386: vector.S
>>> Added Files:
>>>   src/sys/arch/amd64/amd64: amd64_trap.S
>>>   src/sys/arch/i386/i386: i386_trap.S i386_trap_ipkdb.S
>>>
>>> Log Message:
>>> Split these to improve diffability.
>>>
>>>
>>> To generate a diff of this commit:
>>> cvs rdiff -u -r0 -r1.1 src/sys/arch/amd64/amd64/amd64_trap.S
>>> cvs rdiff -u -r1.43 -r1.44 src/sys/arch/amd64/amd64/vector.S
>>> cvs rdiff -u -r0 -r1.1 src/sys/arch/i386/i386/i386_trap.S \
>>> src/sys/arch/i386/i386/i386_trap_ipkdb.S
>>> cvs rdiff -u -r1.61 -r1.62 src/sys/arch/i386/i386/vector.S
>>>
>>> Please note that diffs are not public domain; they are subject to the
>>> copyright notices on the relevant files.
> 


Re: CVS commit: src

2013-08-20 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: joerg
> Date: Mon Aug 19 08:03:34 UTC 2013
> 
> Modified Files:
>   src/include: langinfo.h nl_types.h wchar.h
>   src/lib/libc/include: namespace.h
>   src/lib/libc/locale: nl_langinfo.c wcsftime.c
>   src/lib/libc/nls: Makefile.inc catopen.c
> Removed Files:
>   src/lib/libc/nls: _catclose.c _catgets.c _catopen.c
> 
> Log Message:
> Add nl_langinfo_l, catopen_l and wcsftime_l.

is catopen_l your invention?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.9 -r1.10 src/include/langinfo.h
> cvs rdiff -u -r1.12 -r1.13 src/include/nl_types.h
> cvs rdiff -u -r1.38 -r1.39 src/include/wchar.h
> cvs rdiff -u -r1.166 -r1.167 src/lib/libc/include/namespace.h
> cvs rdiff -u -r1.15 -r1.16 src/lib/libc/locale/nl_langinfo.c
> cvs rdiff -u -r1.3 -r1.4 src/lib/libc/locale/wcsftime.c
> cvs rdiff -u -r1.11 -r1.12 src/lib/libc/nls/Makefile.inc
> cvs rdiff -u -r1.7 -r0 src/lib/libc/nls/_catclose.c \
> src/lib/libc/nls/_catopen.c
> cvs rdiff -u -r1.8 -r0 src/lib/libc/nls/_catgets.c
> cvs rdiff -u -r1.31 -r1.32 src/lib/libc/nls/catopen.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 
> 


Re: CVS commit: src/lib/libc

2013-08-20 Thread YAMAMOTO Takashi
> Module Name:  src
> Committed By: joerg
> Date: Mon Aug 19 22:43:28 UTC 2013
> 
> Modified Files:
>   src/lib/libc/citrus: citrus_lc_ctype.c
>   src/lib/libc/gen: isctype.c
>   src/lib/libc/locale: global_locale.c setlocale_local.h
> 
> Log Message:
> Remove most LC_CTYPE specific parts of locale.cache.

why?

YAMAMOTO Takashi

> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.12 -r1.13 src/lib/libc/citrus/citrus_lc_ctype.c
> cvs rdiff -u -r1.24 -r1.25 src/lib/libc/gen/isctype.c
> cvs rdiff -u -r1.18 -r1.19 src/lib/libc/locale/global_locale.c
> cvs rdiff -u -r1.12 -r1.13 src/lib/libc/locale/setlocale_local.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.


Re: CVS commit: src/lib/libc

2013-08-20 Thread YAMAMOTO Takashi
> On Tue, Aug 20, 2013 at 09:31:18AM +0000, YAMAMOTO Takashi wrote:
>> > Module Name:   src
>> > Committed By:  joerg
>> > Date:  Mon Aug 19 22:43:28 UTC 2013
>> > 
>> > Modified Files:
>> >src/lib/libc/citrus: citrus_lc_ctype.c
>> >src/lib/libc/gen: isctype.c
>> >src/lib/libc/locale: global_locale.c setlocale_local.h
>> > 
>> > Log Message:
>> > Remove most LC_CTYPE specific parts of locale.cache.
>> 
>> why?
> 
> Unlike the locale parts, the cache is currently not invariant and also
> not updated atomically. I am working on fixing that. It is useful for
> run time memory usage to remove redundant fields and reduce the number
> of categories that affect the cache.

for what purpose do you need atomic update?

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src

2013-08-20 Thread YAMAMOTO Takashi
> On Tue, Aug 20, 2013 at 09:30:12AM +0000, YAMAMOTO Takashi wrote:
>> > Module Name:   src
>> > Committed By:  joerg
>> > Date:  Mon Aug 19 08:03:34 UTC 2013
>> > 
>> > Modified Files:
>> >src/include: langinfo.h nl_types.h wchar.h
>> >src/lib/libc/include: namespace.h
>> >src/lib/libc/locale: nl_langinfo.c wcsftime.c
>> >src/lib/libc/nls: Makefile.inc catopen.c
>> > Removed Files:
>> >src/lib/libc/nls: _catclose.c _catgets.c _catopen.c
>> > 
>> > Log Message:
>> > Add nl_langinfo_l, catopen_l and wcsftime_l.
>> 
>> is catopen_l your invention?
> 
> Yes.

can you document the semantics?

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src/lib/libc

2013-08-20 Thread YAMAMOTO Takashi
> On Tue, Aug 20, 2013 at 03:31:01PM +0000, YAMAMOTO Takashi wrote:
>> > On Tue, Aug 20, 2013 at 09:31:18AM +0000, YAMAMOTO Takashi wrote:
>> >> > Module Name:src
>> >> > Committed By:   joerg
>> >> > Date:   Mon Aug 19 22:43:28 UTC 2013
>> >> > 
>> >> > Modified Files:
>> >> > src/lib/libc/citrus: citrus_lc_ctype.c
>> >> > src/lib/libc/gen: isctype.c
>> >> > src/lib/libc/locale: global_locale.c setlocale_local.h
>> >> > 
>> >> > Log Message:
>> >> > Remove most LC_CTYPE specific parts of locale.cache.
>> >> 
>> >> why?
>> > 
>> > Unlike the locale parts, the cache is currently not invariant and also
>> > not updated atomically. I am working on fixing that. It is useful for
>> > run time memory usage to remove redundant fields and reduce the number
>> > of categories that affect the cache.
>> 
>> for what purpose do you need atomic update?
> 
> In a multi-threaded application, calling nl_langinfo or localeconv
> should provide consistent results. It doesn't do that ATM.

- who updates them behind nl_langinfo/localeconv?
- does looking at part_impl directly make them atomic?  how?

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src/lib/libc

2013-08-21 Thread YAMAMOTO Takashi
> On Wed, Aug 21, 2013 at 12:43:58AM +0000, YAMAMOTO Takashi wrote:
>> > On Tue, Aug 20, 2013 at 03:31:01PM +0000, YAMAMOTO Takashi wrote:
>> >> > On Tue, Aug 20, 2013 at 09:31:18AM +, YAMAMOTO Takashi wrote:
>> >> >> > Module Name: src
>> >> >> > Committed By:joerg
>> >> >> > Date:Mon Aug 19 22:43:28 UTC 2013
>> >> >> > 
>> >> >> > Modified Files:
>> >> >> >  src/lib/libc/citrus: citrus_lc_ctype.c
>> >> >> >  src/lib/libc/gen: isctype.c
>> >> >> >  src/lib/libc/locale: global_locale.c setlocale_local.h
>> >> >> > 
>> >> >> > Log Message:
>> >> >> > Remove most LC_CTYPE specific parts of locale.cache.
>> >> >> 
>> >> >> why?
>> >> > 
>> >> > Unlike the locale parts, the cache is currently not invariant and also
>> >> > not updated atomically. I am working on fixing that. It is useful for
>> >> > run time memory usage to remove redundant fields and reduce the number
>> >> > of categories that affect the cache.
>> >> 
>> >> for what purpose do you need atomic update?
>> > 
>> > In a multi-threaded application, calling nl_langinfo or localeconv
>> > should provide consistent results. It doesn't do that ATM.
>> 
>> - who updates them behind nl_langinfo/localeconv?
> 
> setlocale

i don't think it's supposed to work.
is there any standard which mandates it,
or any application which assumes it?

> 
>> - does looking at part_impl directly make them atomic?  how?
> 
> part_impl doesn't change after creation, so changes are visible at the
> point that part_impl is updated. This is the best that can be done
> without using explicit locking. It ensures that at least all data
> from the same category is consistent.

thanks for explanation.
is there anything which needs such a consistency?

YAMAMOTO Takashi

> 
> Joerg


Re: CVS commit: src/sys/ufs/ufs

2014-05-22 Thread YAMAMOTO Takashi
> Indeed rebooting with an updated kernel will give active NFS clients
> problems, but I am not sure we should realy care nor how we could
> possibly avoid this one time issue. We have changed encoding of
> filehandles before (at least once).

it's a bad thing and worth mentioning in release notes.

how about adding a version field so that it won't happen again?

YAMAMOTO Takashi


<    1   2