Re: [GSoC] About the idea: Unicode support in vi
Hi, On Wed, Mar 23, 2011 at 4:36 PM, Zhihao Yuan wrote: > On Wed, Mar 23, 2011 at 11:05 AM, Pan Tsu wrote: >> Zhihao Yuan writes: >>> I'm not sure whether the FreeBSD system administrators (who opens 100 >>> ssh sessions) agree with that to replace the nvi in base system with >>> this one. >> >> Do they expect more features beyond POSIX vi? > > Like multiple windows. This has been discussed y other BSDs before. > For the reference, on the Linux side, busybox do all what an admin would reasonably expect (I mean _all_ the basic userland, not just editing text) in a binary smaller than nvi. Now, it's true that _you_ might not care about size/bloat, at least accept that some do. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [GSoC] About the idea: Unicode support in vi
Hi, On Wed, Mar 23, 2011 at 7:32 PM, Zhihao Yuan wrote: > Among *all* the GNU/Linux distributions I used, they include a vim > compiled in tiny mode (ln -s it to vi), which doubles the size of nvi, > in their base systems. A vim.tiny contains much more features compared > with nvi, but it's not compatible with POSIX vi. > Let's compare the comparable, I don't really care if PCbsd ship vim as its default, but FreeBSD as the base is not only aimed at desktop specifically. So you should take into account that I may want to run FreeBSD on an adm5120 board with 32MB of RAM, without having a text editor consuming too much disk-space/ram. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: New Boot-Loader
Hi, On Mon, Mar 28, 2011 at 1:50 PM, Andrew Duane wrote: > -Original Message- > From: owner-freebsd-hack...@freebsd.org > [mailto:owner-freebsd-hack...@freebsd.org] On Behalf Of Andriy Gapon > Sent: Monday, March 28, 2011 8:00 AM > To: Alexander Best > Cc: FreeBSD Hackers > Subject: Re: New Boot-Loader > > Please note that graphical loaders are not very serial console friendly ;-) > > -- > Andriy Gapon > > > Amen, and we have a whole lot of platforms that are only serial consoles (and > 9600 baud at that). FWIW, an interesting thing, from my point of view, would be to get rid of that 9600bps limitation due to the use of INT14 in the first stage bootloader to have access to more descent serial console speed in early boot stage. > Also, I like the letters instead of numbers for boot options, for those of us > that have known for years that "s" is single user mode, "v" is verbose, > etc > you can also have a thought to the people who are used to use numbers :) - Arnaud > > > ... > Andrew Duane > Juniper Networks > o +1 978 589 0551 > m +1 603-770-7088 > adu...@juniper.net > > > > > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi,
On Wed, Mar 30, 2011 at 8:00 AM, John Baldwin wrote:
> On Tuesday, March 29, 2011 5:20:30 pm m...@freebsd.org wrote:
>> I thought I knew something about how the compiler looks for include
>> files, but now I think maybe I don't know much. :-)
>>
>> So here's what I'm pondering. When I build a library, like e.g. libc,
>> where do the include files get pulled from? They can't (shouldn't) be
>> the ones in /usr/include, but I don't see a -nostdinc like for the
>> kernel. There are -I directives in the Makefile for
>> -I${.CURDIR}/include -I${.CURDIR}/../../include, etc., but that won't
>> remove /usr/include from the search path.
>>
>> I see in the gcc documentation that -I paths are searched before the
>> standards paths. But isn't the lack of -nostdinc a bug (not just for
>> libc, but for any library in /usr/src/lib)? It somewhat feels to me
>> that all of the libraries and binaries in the source distribution
>> should use -nostdinc and include only from the source distribution
>> itself. This isn't always an issue, but for source upgrades it seems
>> crucial, and for a hacker it saves difficulties with having to install
>> headers before re-building.
>>
>> Is that the intent, and it's not fully implemented? How badly would
>> things break if -nostdinc was included in e.g. bsd.lib.mk? (This would
>> break non-base libraries, yes? But as a thought experiment for the
>> base, how far off are we?)
>
> If you are building a library by hand you do want to use the includes from
> /usr/include. I am not sure how we accomplish during buildworld (but we do).
> I think we actually build the compiler in the cross-tools stage such that
> it uses the /usr/include directory under {WORLDTMP} in place of /usr/include
> in the default search path.
>
> Some other folks might be able to verify that (perhaps ru@?).
>
FWIW (I've been hacking around `buildworld' lately), yes, and the
`_includes' stage is responsible to populate ${WORLDTMP}/usr/include.
The same goes for ${WORLDTMP}/usr/lib and `_libraries'. That was with
a 7-stable tree, I'm not sure how clang integrates in all this. The
same way I suppose.
- Arnaud
> --
> John Baldwin
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: make question
Hi, On Thu, Apr 28, 2011 at 11:52 AM, Hartmut Brandt wrote: > I think we can change this, because it would break makefiles that assume > that the entire script is given to the shell in one piece. > I'm not sure to parse that. "We can change it because it would break stuff". That said, if something was to be broken, it would already shows up when using "-j N", and thus should be considered as a bug. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [LIBC] Modfied Version of sscanf
Hi, On Mon, May 2, 2011 at 8:13 AM, Martin Möller wrote: > Hello, > > Thanks for all the replies. > We have so far discovered the following suggetions for the parsing Problem: > Using: > o a tokenizer/parser is too much overhead for such a simple task > o strchr, memchr is too low-level and not elegant enough > o strtok would not even parse (tokenize) this simple example > o a regexp library: How would you solve the problem with a regexp lib ? > > Criteria: > o Receive the value of > o Check the Environment: Is really sourrounded by 'GET ' and > 'HTTP/1.1' ?! > There is plenty of library out there dealing with parsing the code you pointed out. Do you really need to re-invent the wheel ? - Arnaud > I need a function which accepts BNF-style rules. > E.g.: > char resource [512]; > > ret = bnfparse (request, "GET %s HTTP/1.1", resource); > > Ret would be $(NUMBER OF FORMAT SPECIFIERS, successfully handled) +env. > Env would be 1 if the Environment passes, and 0 is not. > > Any comments ? > Best regards, > Martin > > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NFS mount inside jail fails
Hi, On Wed, May 18, 2011 at 10:03 AM, Pawel Jakub Dawidek wrote: > On Tue, May 17, 2011 at 10:17:12PM +0200, Alexander Leidinger wrote: >> On Tue, 17 May 2011 12:56:40 -0700 Sean Bruno >> wrote: >> >> > Silly thing I ran into today. User wanted to NFS mount a dir inside a >> > jail. After I groaned about the security implication of this, I noted >> > that there is a sysctl that looks like it should allow this. Namely, >> > security.jail.mount_allowed. I noted that setting this follows a path >> > that *should* have allowed this silly thing to happen, except that the >> > credentials in the nfsclient were not setup correctly. >> >> As you noticed, this is supposed to allow to mount inside a jail, IF >> the FS you want to mount is marked as secure/safe to do so. Nearly no >> FS is marked as such, as nobody wants to guarantee that it is safe >> (root in a jail should not be able to panic a system by trying to >> mount a corrupt/malicious FS-image) and secure (not possible to get >> elevated access/privileges). >> >> For NFS there is theoretically the problem that the outgoing address on >> requests could be the one of the physical host instead of the IP of the >> jail. If this is true in practice, I do not know. This could be >> the reason why NFS is not marked with VFCF_JAIL. > > It is not marked with VFCF_JAIL, because I just had no time to audit > that it is safe. It might be safe in theory. > > There are some file systems types that can't be securely mounted within > a jail no matter what, like UFS, MSDOFS, EXTFS, XFS, REISERFS, NTFS, > etc. because the user mounting it has access to raw storage and can > corrupt it in a way that it will panic entire system. > This should at least be configurable somehow for people who are using jails for separation and do not care about security. I'd expect that security decision whether or not to allow something is user relevant, not developer relevant. - Arnaud > There are other file systems that don't require access to raw storage > for the user doing the mount and chances are they are safe to mount from > within a jail, like ZFS (user can have access to ZFS datasets, but don't > need access to ZFS pool), NFS, SMBFS, NULLFS, UNIONFS, PROCFS, FDESCFS, > etc. I added VFCF_JAIL flag, so there is general mechanism to mark file > systems as jail-friendly, but back then I only needed it for ZFS. > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi Warner, On Sat, Apr 2, 2011 at 6:49 PM, Warner Losh wrote: > > On Apr 2, 2011, at 1:10 PM, Robert N. M. Watson wrote: > >> On 2 Apr 2011, at 19:47, Warner Losh wrote: >> (2) Working clang/LLVM cross-compile of FreeBSD. This seems like a basic requirement to adopt clang/LLVM, and as far as I'm aware that's not yet a resolved issue? >>> >>> 0 work has been done here to my knowledge. The world view for clang and >>> our in-tree gcc differ which makes it a challenge. >> >> That's disappointing. I seem to recall it's more an issue of our build >> integration with clang/LLVM than an underlying issue in clang/LLVM? > > Yes. The problem isn't hard, the cross compile paradigm is just a little > different. > We (Cambridge) are currently bringing up FreeBSD on a new soft-core 64-bit MIPS platform. We're already using a non-base gcc for our boot loader work, and plan to move to using clang/LLVM later in the year. The base system seems a bit short on detail when it comes to the above, currently. >>> >>> Yes. I've had to add about a dozen changes so far to get close to building >>> with xdev compilers. A similar number are needed to make it easy to >>> configure and add systree support, I think. >> >> Sounds like great progress -- do you think we'll ship 9.0 in a "just works" >> state with regard to this? > > I sure hope so. I'd like to have demoable stuff by BSDcan. > BSDCan has passed, has there been any advance made since that discussion ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi, On Tue, May 31, 2011 at 12:23 PM, Warner Losh wrote: > > On May 22, 2011, at 9:48 PM, Arnaud Lacombe wrote: > >> Hi Warner, >> >> On Sat, Apr 2, 2011 at 6:49 PM, Warner Losh wrote: >>> >>> On Apr 2, 2011, at 1:10 PM, Robert N. M. Watson wrote: >>> >>>> On 2 Apr 2011, at 19:47, Warner Losh wrote: >>>> >>>>>> (2) Working clang/LLVM cross-compile of FreeBSD. This seems like a basic >>>>>> requirement to adopt clang/LLVM, and as far as I'm aware that's not yet >>>>>> a >>>>>> resolved issue? >>>>> >>>>> 0 work has been done here to my knowledge. The world view for clang and >>>>> our in-tree gcc differ which makes it a challenge. >>>> >>>> That's disappointing. I seem to recall it's more an issue of our build >>>> integration with clang/LLVM than an underlying issue in clang/LLVM? >>> >>> Yes. The problem isn't hard, the cross compile paradigm is just a little >>> different. >>> >>>>>> We (Cambridge) are currently bringing up FreeBSD on a new soft-core >>>>>> 64-bit MIPS platform. We're already using a non-base gcc for our boot >>>>>> loader work, and plan to move to using clang/LLVM later in the year. >>>>>> The base system seems a bit short on detail when it comes to the above, >>>>>> currently. >>>>> >>>>> Yes. I've had to add about a dozen changes so far to get close to >>>>> building with xdev compilers. A similar number are needed to make it >>>>> easy to configure and add systree support, I think. >>>> >>>> Sounds like great progress -- do you think we'll ship 9.0 in a "just >>>> works" state with regard to this? >>> >>> I sure hope so. I'd like to have demoable stuff by BSDcan. >>> >> BSDCan has passed, has there been any advance made since that discussion ? > > It is "demonstrable" but not ready to commit to the tree. Needs about 4 > hours of work that I've had trouble scheduling on it due to work getting > busier than I expected. > any chances to have a look at the patch or should I wait for the final commit ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi, On Mon, Jun 6, 2011 at 5:37 PM, Arnaud Lacombe wrote: > Hi, > > On Tue, May 31, 2011 at 12:23 PM, Warner Losh wrote: >> >> On May 22, 2011, at 9:48 PM, Arnaud Lacombe wrote: >> >>> Hi Warner, >>> >>> On Sat, Apr 2, 2011 at 6:49 PM, Warner Losh wrote: >>>> >>>> On Apr 2, 2011, at 1:10 PM, Robert N. M. Watson wrote: >>>> >>>>> On 2 Apr 2011, at 19:47, Warner Losh wrote: >>>>> >>>>>>> (2) Working clang/LLVM cross-compile of FreeBSD. This seems like a >>>>>>> basic >>>>>>> requirement to adopt clang/LLVM, and as far as I'm aware that's not >>>>>>> yet a >>>>>>> resolved issue? >>>>>> >>>>>> 0 work has been done here to my knowledge. The world view for clang and >>>>>> our in-tree gcc differ which makes it a challenge. >>>>> >>>>> That's disappointing. I seem to recall it's more an issue of our build >>>>> integration with clang/LLVM than an underlying issue in clang/LLVM? >>>> >>>> Yes. The problem isn't hard, the cross compile paradigm is just a little >>>> different. >>>> >>>>>>> We (Cambridge) are currently bringing up FreeBSD on a new soft-core >>>>>>> 64-bit MIPS platform. We're already using a non-base gcc for our boot >>>>>>> loader work, and plan to move to using clang/LLVM later in the year. >>>>>>> The base system seems a bit short on detail when it comes to the above, >>>>>>> currently. >>>>>> >>>>>> Yes. I've had to add about a dozen changes so far to get close to >>>>>> building with xdev compilers. A similar number are needed to make it >>>>>> easy to configure and add systree support, I think. >>>>> >>>>> Sounds like great progress -- do you think we'll ship 9.0 in a "just >>>>> works" state with regard to this? >>>> >>>> I sure hope so. I'd like to have demoable stuff by BSDcan. >>>> >>> BSDCan has passed, has there been any advance made since that discussion ? >> >> It is "demonstrable" but not ready to commit to the tree. Needs about 4 >> hours of work that I've had trouble scheduling on it due to work getting >> busier than I expected. >> > any chances to have a look at the patch or should I wait for the final commit > ? > ping ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi, On Sat, Jun 25, 2011 at 2:01 PM, Arnaud Lacombe wrote: > Hi, > > On Mon, Jun 6, 2011 at 5:37 PM, Arnaud Lacombe wrote: >> Hi, >> >> On Tue, May 31, 2011 at 12:23 PM, Warner Losh wrote: >>> >>> On May 22, 2011, at 9:48 PM, Arnaud Lacombe wrote: >>> >>>> Hi Warner, >>>> >>>> On Sat, Apr 2, 2011 at 6:49 PM, Warner Losh wrote: >>>>> >>>>> On Apr 2, 2011, at 1:10 PM, Robert N. M. Watson wrote: >>>>> >>>>>> On 2 Apr 2011, at 19:47, Warner Losh wrote: >>>>>> >>>>>>>> (2) Working clang/LLVM cross-compile of FreeBSD. This seems like a >>>>>>>> basic >>>>>>>> requirement to adopt clang/LLVM, and as far as I'm aware that's not >>>>>>>> yet a >>>>>>>> resolved issue? >>>>>>> >>>>>>> 0 work has been done here to my knowledge. The world view for clang >>>>>>> and our in-tree gcc differ which makes it a challenge. >>>>>> >>>>>> That's disappointing. I seem to recall it's more an issue of our build >>>>>> integration with clang/LLVM than an underlying issue in clang/LLVM? >>>>> >>>>> Yes. The problem isn't hard, the cross compile paradigm is just a little >>>>> different. >>>>> >>>>>>>> We (Cambridge) are currently bringing up FreeBSD on a new soft-core >>>>>>>> 64-bit MIPS platform. We're already using a non-base gcc for our boot >>>>>>>> loader work, and plan to move to using clang/LLVM later in the year. >>>>>>>> The base system seems a bit short on detail when it comes to the >>>>>>>> above, currently. >>>>>>> >>>>>>> Yes. I've had to add about a dozen changes so far to get close to >>>>>>> building with xdev compilers. A similar number are needed to make it >>>>>>> easy to configure and add systree support, I think. >>>>>> >>>>>> Sounds like great progress -- do you think we'll ship 9.0 in a "just >>>>>> works" state with regard to this? >>>>> >>>>> I sure hope so. I'd like to have demoable stuff by BSDcan. >>>>> >>>> BSDCan has passed, has there been any advance made since that discussion ? >>> >>> It is "demonstrable" but not ready to commit to the tree. Needs about 4 >>> hours of work that I've had trouble scheduling on it due to work getting >>> busier than I expected. >>> >> any chances to have a look at the patch or should I wait for the final >> commit ? >> > ping ? > ping ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
buf_ring(9) API precisions
Hi Kip,
I've got a few question about the buf_ring(9) API.
1) what means the 'drbr_' prefix. I can guess the two last letter, 'b'
and 'r', for Buffer Ring, but what about 'd' and 'r' ?
2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
struct buf_ring {
volatile uint32_t br_prod_head;
volatile uint32_t br_prod_tail;
int br_prod_size;
int br_prod_mask;
uint64_tbr_drops;
uint64_tbr_prod_bufs;
uint64_tbr_prod_bytes;
/*
* Pad out to next L2 cache line
*/
uint64_t_pad0[11];
volatile uint32_t br_cons_head;
volatile uint32_t br_cons_tail;
int br_cons_size;
int br_cons_mask;
/*
* Pad out to next L2 cache line
*/
uint64_t_pad1[14];
#ifdef DEBUG_BUFRING
struct mtx *br_lock;
#endif
void*br_ring[0];
};
Why are you making an MD guess, the amount of padding to fit the size
of a cache line, in MI API ? Strangely enough, you did not make this
assumption in, say r205488 (picked randomly).
Thanks in advance,
- Arnaud
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Include file search path
Hi, On Fri, Jul 8, 2011 at 5:16 PM, Arnaud Lacombe wrote: > Hi, > > On Sat, Jun 25, 2011 at 2:01 PM, Arnaud Lacombe wrote: >> Hi, >> >> On Mon, Jun 6, 2011 at 5:37 PM, Arnaud Lacombe wrote: >>> Hi, >>> >>> On Tue, May 31, 2011 at 12:23 PM, Warner Losh wrote: >>>> >>>> On May 22, 2011, at 9:48 PM, Arnaud Lacombe wrote: >>>> >>>>> Hi Warner, >>>>> >>>>> On Sat, Apr 2, 2011 at 6:49 PM, Warner Losh wrote: >>>>>> >>>>>> On Apr 2, 2011, at 1:10 PM, Robert N. M. Watson wrote: >>>>>> >>>>>>> On 2 Apr 2011, at 19:47, Warner Losh wrote: >>>>>>> >>>>>>>>> (2) Working clang/LLVM cross-compile of FreeBSD. This seems like a >>>>>>>>> basic >>>>>>>>> requirement to adopt clang/LLVM, and as far as I'm aware that's not >>>>>>>>> yet a >>>>>>>>> resolved issue? >>>>>>>> >>>>>>>> 0 work has been done here to my knowledge. The world view for clang >>>>>>>> and our in-tree gcc differ which makes it a challenge. >>>>>>> >>>>>>> That's disappointing. I seem to recall it's more an issue of our build >>>>>>> integration with clang/LLVM than an underlying issue in clang/LLVM? >>>>>> >>>>>> Yes. The problem isn't hard, the cross compile paradigm is just a >>>>>> little different. >>>>>> >>>>>>>>> We (Cambridge) are currently bringing up FreeBSD on a new soft-core >>>>>>>>> 64-bit MIPS platform. We're already using a non-base gcc for our >>>>>>>>> boot loader work, and plan to move to using clang/LLVM later in the >>>>>>>>> year. The base system seems a bit short on detail when it comes to >>>>>>>>> the above, currently. >>>>>>>> >>>>>>>> Yes. I've had to add about a dozen changes so far to get close to >>>>>>>> building with xdev compilers. A similar number are needed to make it >>>>>>>> easy to configure and add systree support, I think. >>>>>>> >>>>>>> Sounds like great progress -- do you think we'll ship 9.0 in a "just >>>>>>> works" state with regard to this? >>>>>> >>>>>> I sure hope so. I'd like to have demoable stuff by BSDcan. >>>>>> >>>>> BSDCan has passed, has there been any advance made since that discussion ? >>>> >>>> It is "demonstrable" but not ready to commit to the tree. Needs about 4 >>>> hours of work that I've had trouble scheduling on it due to work getting >>>> busier than I expected. >>>> >>> any chances to have a look at the patch or should I wait for the final >>> commit ? >>> >> ping ? >> > ping ? > ping, Warner ? FWIW, I guess that in 4 months, either you had time to finish the patch, or I could have found to time to complete it myself based on your incomplete version. I'd really be interested to benchmark recent, decent, compiler. Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: buf_ring(9) API precisions
Hi,
On Thu, Sep 15, 2011 at 5:21 PM, Robert Watson wrote:
> On Thu, 15 Sep 2011, K. Macy wrote:
>
>>> Why are you making an MD guess, the amount of padding to fit the size of
>>> a cache line, in MI API ? Strangely enough, you did not make this assumption
>>> in, say r205488 (picked randomly).
>>
>> It has been several years, and I haven't done any work in svn in over a
>> year, I don't remember. I probably meant to refine it in a later iteration.
>>
>> If you would like to send me a patch addressing this I'd be more than
>> happy to apply it if appropriate. Otherwise, I will deal with it some time
>> after 9 settles.
>>
>> Thanks for pointing this out.
>
> I'm not sure if gcc (and friends) allow __aligned(CACHE_LINE_SIZE) to be
> used on individual elements of a struct (causing appropriate padding to be
> added), but that may be one option here.
>
It definitively does, it is used in several structure in the tree.
Attached a patch to convert buf_ring(9), however, I'm not a huge fan
of dedicating a full cache line to the lock pointer.
- Arnaud
> Of course, that introduces a
> further alignment requirement on the struct itself, so a moderate amount of
> care would need to be used.
>
> Robert
>
diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h
index 57e42e5..01f399e 100644
--- a/sys/sys/buf_ring.h
+++ b/sys/sys/buf_ring.h
@@ -49,25 +49,17 @@ struct buf_ring {
uint64_t br_drops;
uint64_t br_prod_bufs;
uint64_t br_prod_bytes;
- /*
- * Pad out to next L2 cache line
- */
- uint64_t _pad0[11];
- volatile uint32_t br_cons_head;
+ volatile uint32_t br_cons_head __aligned(CACHE_LINE_SIZE);
volatile uint32_t br_cons_tail;
int br_cons_size;
int br_cons_mask;
- /*
- * Pad out to next L2 cache line
- */
- uint64_t _pad1[14];
#ifdef DEBUG_BUFRING
- struct mtx *br_lock;
+ struct mtx *br_lock __aligned(CACHE_LINE_SIZE);
#endif
- void *br_ring[0];
-};
+ void *br_ring[0] __aligned(CACHE_LINE_SIZE);
+} __aligned(CACHE_LINE_SIZE);
/*
* multi-producer safe lock-free ring buffer enqueue
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: buf_ring(9) API precisions
Hi,
On Wed, Sep 14, 2011 at 10:53 PM, Arnaud Lacombe wrote:
> Hi Kip,
>
> I've got a few question about the buf_ring(9) API.
>
> 1) what means the 'drbr_' prefix. I can guess the two last letter, 'b'
> and 'r', for Buffer Ring, but what about 'd' and 'r' ?
>
> 2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
>
> struct buf_ring {
> volatile uint32_t br_prod_head;
> volatile uint32_t br_prod_tail;
> int br_prod_size;
> int br_prod_mask;
> uint64_t br_drops;
> uint64_t br_prod_bufs;
> uint64_t br_prod_bytes;
shouldn't those 3 fields be updated atomically, especially on 32bits
platforms ? That might pose a problem as, AFAIK, FreeBSD do not have
MI 64bits atomics operations...
- Arnaud
> /*
> * Pad out to next L2 cache line
> */
> uint64_t _pad0[11];
>
> volatile uint32_t br_cons_head;
> volatile uint32_t br_cons_tail;
> int br_cons_size;
> int br_cons_mask;
>
> /*
> * Pad out to next L2 cache line
> */
> uint64_t _pad1[14];
> #ifdef DEBUG_BUFRING
> struct mtx *br_lock;
> #endif
> void *br_ring[0];
> };
>
> Why are you making an MD guess, the amount of padding to fit the size
> of a cache line, in MI API ? Strangely enough, you did not make this
> assumption in, say r205488 (picked randomly).
>
> Thanks in advance,
> - Arnaud
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Sat, Sep 17, 2011 at 4:32 PM, YongHyeon PYUN wrote: > On Sat, Sep 17, 2011 at 11:57:10AM +0430, Hooman Fazaeli wrote: >> Hi list, >> >> The data sheet for intel 82576 advertises IP TX/RX checksum offload >> but the driver does not set CSUM_IP in ifp->if_hwassist. Does this mean that >> driver (and chip) do not support IP TX checksum offload or the support for >> TX is not yet included in the driver? > > After reading this mail, I checked em(4)/lem(4) code and noticed > these drivers removed CSUM_IP capability as well. igb(4) didn't > support CSUM_IP from day 1 but em(4)/lem(4) used to take advantage > of IP checksum offloading capability. > Given that these drivers share many code with Linux, Jack may know > the details and why IP checksum offloading code was removed. Note, > Linux does not use IP checksum offloading so I guess this could be > oversight in shared code. > > BTW, hackers may not be right ML to post this kind of post. > CCed to jfv@, the driver maintainer. > This is slightly off-topic, but still.. FWIW, I'm not really impressed by what chips claim to support vs. what has been implemented in the driver. As per the product brief, the 82574 supports multiqueue (at least should support up to 5 MSI-X vectors, 2 RX and 2 TX), but this support was removed from em(4) in mid-2010[0]. Also, the 82571 and the 82574 should also support header split, but this is currently only implemented in igb(4), not em(4). - Arnaud [0]: the commit message say "performance was not good", but it is not the driver's developer to decide whether or not a feature is good or not. The developer's job is to implement the chip capabilities, and let it to the user to enable or disable the capabilities. At best, the developer can decide whether or not to enable the feature by default. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Sun, Sep 18, 2011 at 5:06 PM, Luigi Rizzo wrote: > On Sun, Sep 18, 2011 at 03:19:46PM -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Sat, Sep 17, 2011 at 4:32 PM, YongHyeon PYUN wrote: >> > On Sat, Sep 17, 2011 at 11:57:10AM +0430, Hooman Fazaeli wrote: >> >> Hi list, >> >> >> >> The data sheet for intel 82576 advertises IP TX/RX checksum offload >> >> but the driver does not set CSUM_IP in ifp->if_hwassist. Does this mean >> >> that >> >> driver (and chip) do not support IP TX checksum offload or the support for >> >> TX is not yet included in the driver? > ... >> This is slightly off-topic, but still.. >> >> FWIW, I'm not really impressed by what chips claim to support vs. what >> has been implemented in the driver. As per the product brief, the > ... >> [0]: the commit message say "performance was not good", but it is not >> the driver's developer to decide whether or not a feature is good or >> not. The developer's job is to implement the chip capabilities, and >> let it to the user to enable or disable the capabilities. At best, the >> developer can decide whether or not to enable the feature by default. > > actually, this is a perfect example where the developer has done the > right thing: implemented the feature, verified that performance is bad, > hence presumably removed support for the feature from the code (which also > means that the normal code path will run faster because there are no > run-time decisions to be made). > > "optional" features are often costly even when disabled. > I forgot to mention that in this case, the code full of EM_MULTIQUEUE's #ifdef and shared code is still fully compatible with the multiqueue's architecture. The only thing removed is a conditional and an assignation in the driver's attachment which was enabling the feature, ie. the cost you point out is still paid today, without any benefit. Now I might also openly question the test method used by the folks at Intel, just seeing how much issue I've had with the driver (I still have for some, even if not driver related), which have not been reproduced there. Finally, when someone say "performance are better that way", the first thing I'd be tempted to ask is: "What is your test ? How did you collects the numbers ? How did you reach the conclusion ?". None of this stuff is public. regards, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Sun, Sep 18, 2011 at 10:01 PM, Luigi Rizzo wrote: > On Sun, Sep 18, 2011 at 06:05:33PM -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Sun, Sep 18, 2011 at 5:06 PM, Luigi Rizzo wrote: >> > On Sun, Sep 18, 2011 at 03:19:46PM -0400, Arnaud Lacombe wrote: >> >> Hi, >> >> >> >> On Sat, Sep 17, 2011 at 4:32 PM, YongHyeon PYUN wrote: >> >> > On Sat, Sep 17, 2011 at 11:57:10AM +0430, Hooman Fazaeli wrote: >> >> >> Hi list, >> >> >> >> >> >> The data sheet for intel 82576 advertises IP TX/RX checksum offload >> >> >> but the driver does not set CSUM_IP in ifp->if_hwassist. Does this >> >> >> mean that >> >> >> driver (and chip) do not support IP TX checksum offload or the support >> >> >> for >> >> >> TX is not yet included in the driver? >> > ... >> >> This is slightly off-topic, but still.. >> >> >> >> FWIW, I'm not really impressed by what chips claim to support vs. what >> >> has been implemented in the driver. As per the product brief, the >> > ... >> >> [0]: the commit message say "performance was not good", but it is not >> >> the driver's developer to decide whether or not a feature is good or >> >> not. The developer's job is to implement the chip capabilities, and >> >> let it to the user to enable or disable the capabilities. At best, the >> >> developer can decide whether or not to enable the feature by default. >> > >> > actually, this is a perfect example where the developer has done the >> > right thing: implemented the feature, verified that performance is bad, >> > hence presumably removed support for the feature from the code (which also >> > means that the normal code path will run faster because there are no >> > run-time decisions to be made). >> > >> > "optional" features are often costly even when disabled. >> > >> I forgot to mention that in this case, the code full of >> EM_MULTIQUEUE's #ifdef and shared code is still fully compatible with >> the multiqueue's architecture. The only thing removed is a conditional >> and an assignation in the driver's attachment which was enabling the >> feature, ie. the cost you point out is still paid today, without any >> benefit. > > the above suggests that you have a wonderful opportunity: with just > a little bit of time and effort you should be able to complete/re-enable > the missing code, run tests that you believe significant (given > your statement below) and prove or disprove the comment about > performance. > Which I did about a week ago, to finally discover that the NIC only had only 3 MSI-X vectors configured in its EEPROM[0], and thus the MSI-X PCI capability field ends up also with being assigned with those 3 vectors. However, the 82574 datasheet clearly say that up-to 5 vector can be configured, but I obviously did not find the magic trick to make it so. Maybe I'll find some time and try to reprogram the EEPROM. Beside that, it was clear that the old multiqueue did not support only 3 vector being available and thus fell back on MSI. It is not clear in jfv@'s comment whether he really tested multiqueue, or did he test the fall-back MSI mode. As the PCI spec is not public, I've not been able to find out from the few public datasheet how the PCI MSI-X capability field is first programmed. I'd assume that the BIOS is using the data in the NVM to program it at power up. - Arnaud [0]: at least, the MSI_X_NUM field of the NVM at offset 0x1b is 2, thus 3 vectors. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: buf_ring(9) API precisions
Hi,
On Fri, Sep 16, 2011 at 10:41 AM, K. Macy wrote:
> On Fri, Sep 16, 2011 at 3:02 AM, Arnaud Lacombe wrote:
>> Hi,
>>
>> On Wed, Sep 14, 2011 at 10:53 PM, Arnaud Lacombe wrote:
>>> Hi Kip,
>>>
>>> I've got a few question about the buf_ring(9) API.
>>>
>>> 1) what means the 'drbr_' prefix. I can guess the two last letter, 'b'
>>> and 'r', for Buffer Ring, but what about 'd' and 'r' ?
>>>
>>> 2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
>>>
>>> struct buf_ring {
>>> volatile uint32_t br_prod_head;
>>> volatile uint32_t br_prod_tail;
>>> int br_prod_size;
>>> int br_prod_mask;
>>> uint64_t br_drops;
>>> uint64_t br_prod_bufs;
>>> uint64_t br_prod_bytes;
>> shouldn't those 3 fields be updated atomically, especially on 32bits
>> platforms ? That might pose a problem as, AFAIK, FreeBSD do not have
>> MI 64bits atomics operations...
>
> Between the point at which br_prod_tail == prod_head and when we
> update br_prod_tail to point to prod_next we are the exclusive owners
> of the fields in buf_ring. That is why we wait for any other
> enqueueing threads to update br_prod_tail to point to prod_head before
> continuing.
>
How do you enforce ordering ? I do not see anything particular
forbidding the `br->br_prod_tail' to be committed first, leading other
thread to believe they have access to the statistics, while the other
thread has not yet committed its change.
Thanks,
- Arnaud
> Cheers
>
> /*
> * If there are other enqueues in progress
> * that preceeded us, we need to wait for them
> * to complete
> */
> while (br->br_prod_tail != prod_head)
> cpu_spinwait();
> br->br_prod_bufs++;
> br->br_prod_bytes += nbytes;
> br->br_prod_tail = prod_next;
> critical_exit();
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Mon, Sep 19, 2011 at 12:29 AM, Jack Vogel wrote: > [...] > If you notice, the Linux driver did not enable multiqueue on the hardware > either, so do you think a whole department of software engineers backed > by the hardware engineers who designed the damn thing might have had > a reason? > Let's find out :-) In the mean time, as a hacker, I'd be interested to know why the NVM ships with MSI_X_NUM == 2 at offset 0x1b when the datasheet say it does support up value to 4 ? This might be as simple as "the hardware needs special wiring to support 5 vectors.", or whatever... > IN FACT, as I have a bit more freedom with FreeBSD, I went ahead and > tried it for a while just because I could, implementing the code was not > difficult. Over time however that code proved to be a source of instability > and thus was disabled. > It would be interesting for the records to have this information public, especially the kind of instability seen. > I have heard a rumor that the Linux crew may actually be trying a second > time to make it work, and that might give me cause to look at it again too, > but its not clear if I'll have time with other priorities. > ack. Now, what about header-split support in em(4) ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Mon, Sep 19, 2011 at 5:28 AM, Adrian Chadd wrote: > Arnaud (and others), > > Liaising with vendors is not an easy task. The reason why Intel (and > other vendors) don't supply detailed history and reasoning for their > development efforts is that their engineers are likely tasked with > "making it work" versus "writing lots of stuff down for public > release." In some instances, the vendor support of FreeBSD (and "free" > open source in general) is done as a side-project by some of the > engineers inside the company. > > So in this case, you may find that Jack and the other engineers at > Intel just don't have the time or resources to dedicate the kinds of > feedback and support you seem to be after. He and others likely have a > huge set of tasks to do at work and none of them officially include > "support FreeBSD/Linux developers by providing detailed feedback and > assistance." So whenever Jack pops up to help out, he's likely doing > it in his spare time. :-) > Yes, and he seems to really like to waste his spare time by repeating me for two months to increase `kern.ipc.nmbclusters' to fix issue I was seeing, when the code was clearly buggy, even when I sent him patchs fixing issues. That's sure a very efficient way of managing time. - Arnaud > Developers can and will disable or remove functionality which is > problematic because they don't have the time or resources to support > it. Users may wish to turn on unsupported features and then will > complain loudly when they don't work; even giving up and moving to > another piece of equipment because of perceived issues. I agree that > it would be nice if the developers included _all_ features, > unsupported or not, so that developers can choose to work on them if > they wish. It however is a trade-off between trying to provide > developers with more useful things to tinker with and not increasing > support load from users (and other developers) who seek to use > incomplete features. > > I hope this helps. > > > Adrian > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: intel checksum offload
Hi, On Mon, Sep 19, 2011 at 12:59 PM, YongHyeon PYUN wrote: > On Mon, Sep 19, 2011 at 10:17:22AM -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Mon, Sep 19, 2011 at 5:28 AM, Adrian Chadd wrote: >> > Arnaud (and others), >> > >> > Liaising with vendors is not an easy task. The reason why Intel (and >> > other vendors) don't supply detailed history and reasoning for their >> > development efforts is that their engineers are likely tasked with >> > "making it work" versus "writing lots of stuff down for public >> > release." In some instances, the vendor support of FreeBSD (and "free" >> > open source in general) is done as a side-project by some of the >> > engineers inside the company. >> > >> > So in this case, you may find that Jack and the other engineers at >> > Intel just don't have the time or resources to dedicate the kinds of >> > feedback and support you seem to be after. He and others likely have a >> > huge set of tasks to do at work and none of them officially include >> > "support FreeBSD/Linux developers by providing detailed feedback and >> > assistance." So whenever Jack pops up to help out, he's likely doing >> > it in his spare time. :-) >> > >> Yes, and he seems to really like to waste his spare time by repeating >> me for two months to increase `kern.ipc.nmbclusters' to fix issue I >> was seeing, when the code was clearly buggy, even when I sent him >> patchs fixing issues. >> > > If you think you encountered a driver bug, could you share it with > us? I didn't closely follow em(4)/lem(4)/igb(4) changes for a long > time so I'm not sure whether I can come up with reasonable fix for > the issue but I may be able to help you. > They ended up being fixed, thanks :-) Now, up until next week, we were struggle on the mbuf corruption I reported on freebsd-net@, but I do not think it is a driver issue per-se. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: my git development snapshot(s)
Hi, [adding Fabien Thomas to the CC: list] On Sun, Sep 18, 2011 at 5:14 AM, Andriy Gapon wrote: > > Just decided to follow the global trends and trying to throw all of my > local/private changes at you in hope that the "crowd-sourcing magic" might > somehow happen :-) This seems definitely easier than carefully producing the > patch files and keeping them up-to-date. > > So, my newly cloned gitorious repository: > https://gitorious.org/~avg/freebsd/avgbsd > I see that this tree is a clone of: https://gitorious.org/freebsd/freebsd One issue I've got with that tree, is that it is not usable for any serious work because of the following: % git merge-base origin/master origin/svn_stable_8 % echo $? 1 The whole FreeBSD 8.x branch is on its own. However: % git merge-base origin/master origin/svn_stable_7 bdad8b50a25980737a8e00047f41fc1e588fc9e5 % git show bdad8b50a25980737a8e00047f41fc1e588fc9e5 commit bdad8b50a25980737a8e00047f41fc1e588fc9e5 Author: obrien Date: Wed Oct 10 16:59:14 2007 + For 7.0 make the shared lib "version" '3'. Approved by:re(kensmith) git-svn-id: svn://svn.freebsd.org/base/head@172505 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f So something should have gone wrong when the mirror was created with the 8-STABLE branch as its root commit is: commit 47c9e1cbe6e19ab62a112f67fa4baf081aa213be Author: jfv Date: Fri Sep 4 22:37:03 2009 + This patch seperates the control of header split from LRO (which it was previously dependent on), LRO gets turned off when bridging but its been found that header split is still a performance win in that case. Secondly, there was some interface specific control in stats code that has been missing, and a logic error that resulted in bogus reporting. Thanks to Manish and John of LineRateSystems for the report and help in this code. Approved by: re git-svn-id: http://svn.freebsd.org/base/stable/8@196843 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f which should rather be: commit d4a6bb6e18d2601fe08d76be4610ae4a303574b2 Author: ache Date: Fri Jan 21 13:31:02 2005 + Whitespace/style tweaking of prev. commit. Noted by: bde git-svn-id: svn://svn.freebsd.org/base/head@140577 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Regards, - Arnaud > And the first branch of interest: > https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 > > This is a snapshot of almost all of my local changes to the FreeBSD src tree. > Please note that the branch is not intended to be public in the git sense. > That > is, there will be no linear history - I periodically rebase my changes on top > of > the svn head and also frequently reshuffle/merge/split commits. > The snapshot is not tidied up, there are quite a few commits that should be > merged into other commits, some commit messages are not accurate/pretty, etc. > The older the commits, the more mature they are supposed to be. > > Based on the above, no new commits are expected to this snapshot branch. > I will produce new snapshot branches from time to time. > > I am posting this information to this list initially, later I plan to share > the > code with the wider audience e.g. via hackers@. > > P.S. This code sharing is made easier for me by git, Gitorious and "git rebase > --onto" in particular. Thanks to Fabien Thomas for the initial FreeBSD clone > repository at Gitorious! > > -- > Andriy Gapon > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: my git development snapshot(s)
Hi, On Tue, Sep 20, 2011 at 4:58 AM, Fabien Thomas wrote: > > On Sep 20, 2011, at 6:58 AM, Arnaud Lacombe wrote: > >> Hi, >> >> [adding Fabien Thomas to the CC: list] >> >> On Sun, Sep 18, 2011 at 5:14 AM, Andriy Gapon wrote: >>> >>> Just decided to follow the global trends and trying to throw all of my >>> local/private changes at you in hope that the "crowd-sourcing magic" might >>> somehow happen :-) This seems definitely easier than carefully producing >>> the >>> patch files and keeping them up-to-date. >>> >>> So, my newly cloned gitorious repository: >>> https://gitorious.org/~avg/freebsd/avgbsd >>> >> I see that this tree is a clone of: >> >> https://gitorious.org/freebsd/freebsd >> >> One issue I've got with that tree, is that it is not usable for any >> serious work because of the following: > > I'm working seriously with it :D > he, I guess we do not have the same kind of workflow :) In my case, I was trying a tree from 7-STABLE into 8-STABLE. >> >> % git merge-base origin/master origin/svn_stable_8 >> % echo $? >> 1 >> > That's strange: > # git merge-base origin/master origin/svn_stable_8 > 08b4e774d45547b6476807eacbfc1d2b5aab02e2 > > I've tried from fresh fresh clone with same result. > You will see this issue online too: https://gitorious.org/freebsd/freebsd/commits/svn_release_8_0_0?page=8 https://gitorious.org/freebsd/freebsd/commits/svn_release_8_1_0?page=82 https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_0?page=8 https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_1?page=82 There is no history before commit 47c9e1c. https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_2?page=3 There is no history before commit aca2adc. Now, compare with 7.x branches: https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_0?page=776 https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_1?page=858 https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_3?page=928 https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_4?page=963 All have exactly the same root commit: d4a6bb6. I'm trying a new clone now, I'll let you know the result... - Arnaud >> The whole FreeBSD 8.x branch is on its own. However: >> >> % git merge-base origin/master origin/svn_stable_7 >> bdad8b50a25980737a8e00047f41fc1e588fc9e5 >> >> % git show bdad8b50a25980737a8e00047f41fc1e588fc9e5 >> commit bdad8b50a25980737a8e00047f41fc1e588fc9e5 >> Author: obrien >> Date: Wed Oct 10 16:59:14 2007 + >> >> For 7.0 make the shared lib "version" '3'. >> Approved by: re(kensmith) >> >> git-svn-id: svn://svn.freebsd.org/base/head@172505 >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f >> >> So something should have gone wrong when the mirror was created with >> the 8-STABLE branch as its root commit is: > > # git log origin/svn_stable_8 > commit 46c8a8db6dae5c885c7c1de5d39904566cc19685 > Author: ume > Date: Sun Sep 13 11:34:33 2009 + > > MFC r196475: > - Add AS lookup functionality to traceroute6(8) as well. > - Support for IPv6 transport for AS lookup. > - Introduce $RA_SERVER to set whois server. > - Support for 4 byte ASN. > - ANSIfy function declaration in as.c. > > Approved by: re (kib) > > > git-svn-id: http://svn.freebsd.org/base/stable/8@197158 > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f > >> >> commit 47c9e1cbe6e19ab62a112f67fa4baf081aa213be >> Author: jfv >> Date: Fri Sep 4 22:37:03 2009 + >> >> This patch seperates the control of header split from LRO (which it >> was previously dependent on), LRO gets turned off when bridging but >> its been found that header split is still a performance win in that case. >> >> Secondly, there was some interface specific control in stats code that >> has been missing, and a logic error that resulted in bogus reporting. >> Thanks to Manish and John of LineRateSystems for the report and help in >> this code. >> >> Approved by: re >> >> git-svn-id: http://svn.freebsd.org/base/stable/8@196843 >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f >> >> which should rather be: >> >> commit d4a6bb6e18d2601fe08d76be4610ae4a303574b2 >> Author: ache >> Date: Fri Jan 21 13:31:02 2005 + >> >> Whitespace/style tweaking of prev. commit. >> >> Noted by: bde >> >> git-svn-id: svn://svn.freebsd.org/base/head@140577 >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f &
Re: my git development snapshot(s)
Hi, On Tue, Sep 20, 2011 at 10:11 AM, Arnaud Lacombe wrote: > Hi, > > On Tue, Sep 20, 2011 at 4:58 AM, Fabien Thomas > wrote: >> >> On Sep 20, 2011, at 6:58 AM, Arnaud Lacombe wrote: >> >>> Hi, >>> >>> [adding Fabien Thomas to the CC: list] >>> >>> On Sun, Sep 18, 2011 at 5:14 AM, Andriy Gapon wrote: >>>> >>>> Just decided to follow the global trends and trying to throw all of my >>>> local/private changes at you in hope that the "crowd-sourcing magic" might >>>> somehow happen :-) This seems definitely easier than carefully producing >>>> the >>>> patch files and keeping them up-to-date. >>>> >>>> So, my newly cloned gitorious repository: >>>> https://gitorious.org/~avg/freebsd/avgbsd >>>> >>> I see that this tree is a clone of: >>> >>> https://gitorious.org/freebsd/freebsd >>> >>> One issue I've got with that tree, is that it is not usable for any >>> serious work because of the following: >> >> I'm working seriously with it :D >> > he, I guess we do not have the same kind of workflow :) In my case, I > was trying a tree from 7-STABLE into 8-STABLE. > >>> >>> % git merge-base origin/master origin/svn_stable_8 >>> % echo $? >>> 1 >>> >> That's strange: >> # git merge-base origin/master origin/svn_stable_8 >> 08b4e774d45547b6476807eacbfc1d2b5aab02e2 >> >> I've tried from fresh fresh clone with same result. >> > You will see this issue online too: > > https://gitorious.org/freebsd/freebsd/commits/svn_release_8_0_0?page=8 > https://gitorious.org/freebsd/freebsd/commits/svn_release_8_1_0?page=82 > https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_0?page=8 > https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_1?page=82 > > There is no history before commit 47c9e1c. > > https://gitorious.org/freebsd/freebsd/commits/svn_releng_8_2?page=3 > > There is no history before commit aca2adc. > > Now, compare with 7.x branches: > > https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_0?page=776 > https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_1?page=858 > https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_3?page=928 > https://gitorious.org/freebsd/freebsd/commits/svn_releng_7_4?page=963 > > All have exactly the same root commit: d4a6bb6. > > I'm trying a new clone now, I'll let you know the result... > Still the same issue: % git clone git://gitorious.org/freebsd/freebsd.git Cloning into freebsd... remote: Counting objects: 639377, done. remote: Compressing objects: 100% (159732/159732), done. remote: Total 639377 (delta 489007), reused 619601 (delta 470781) Receiving objects: 100% (639377/639377), 339.01 MiB | 382 KiB/s, done. Resolving deltas: 100% (489007/489007), done. % cd freebsd freebsd/% git merge-base origin/master origin/svn_stable_8 freebsd/% echo $? 1 freebsd/% git --version git version 1.7.4.1 freebsd/% git log --pretty=oneline origin/svn_releng_7_0 | wc -l 23276 freebsd/% git log --pretty=oneline origin/svn_releng_7_1 | wc -l 25731 freebsd/% git log --pretty=oneline origin/svn_releng_7_2 | wc -l 26620 freebsd/% git log --pretty=oneline origin/svn_releng_7_3 | wc -l 27826 freebsd/% git log --pretty=oneline origin/svn_releng_7_4 | wc -l 28872 freebsd/% git log --pretty=oneline origin/svn_releng_8_0 | wc -l 238 freebsd/% git log --pretty=oneline origin/svn_releng_8_1 | wc -l 2446 freebsd/% git log --pretty=oneline origin/svn_releng_8_2 | wc -l 77 freebsd/% git log --pretty=oneline origin/master | wc -l 52252 Beside that, this repository is the only one with which I've got problem. - Arnaud > >>> The whole FreeBSD 8.x branch is on its own. However: >>> >>> % git merge-base origin/master origin/svn_stable_7 >>> bdad8b50a25980737a8e00047f41fc1e588fc9e5 >>> >>> % git show bdad8b50a25980737a8e00047f41fc1e588fc9e5 >>> commit bdad8b50a25980737a8e00047f41fc1e588fc9e5 >>> Author: obrien >>> Date: Wed Oct 10 16:59:14 2007 + >>> >>> For 7.0 make the shared lib "version" '3'. >>> Approved by: re(kensmith) >>> >>> git-svn-id: svn://svn.freebsd.org/base/head@172505 >>> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f >>> >>> So something should have gone wrong when the mirror was created with >>> the 8-STABLE branch as its root commit is: >> >> # git log origin/svn_stable_8 >> commit 46c8a8db6dae5c885c7c1de5d39904566cc19685 >> Author: ume >> Date:
Re: buf_ring(9) API precisions
Hi,
On Mon, Sep 19, 2011 at 8:46 AM, K. Macy wrote:
> If the value lags next by one then it is ours. This rule applies to
> all callers so the rule holds consistently.
>
I think you do not understand what I mean, which is that the following:
while (br->br_prod_tail != prod_head)
cpu_spinwait();
br->br_prod_bufs++;
br->br_prod_bytes += nbytes;
br->br_prod_tail = prod_next;
critical_exit();
at runtime, can be seen, memory-wise as:
while (br->br_prod_tail != prod_head)
cpu_spinwait();
br->br_prod_tail = prod_next;
br->br_prod_bufs++;
br->br_prod_bytes += nbytes;
critical_exit();
That is, there is no memory barrier to enforce completion of the
load/increment/store/load/load/addition/store operations before
updating what other thread spin on. Yes, `br_prod_tail' is marked
`volatile', but there is no guarantee that it will not be re-ordered
wrt. non-volatile write (to `br_prod_bufs' and `br_prod_bytes').
- Arnaud
> On Mon, Sep 19, 2011 at 5:53 AM, Arnaud Lacombe wrote:
>> Hi,
>>
>> On Fri, Sep 16, 2011 at 10:41 AM, K. Macy wrote:
>>> On Fri, Sep 16, 2011 at 3:02 AM, Arnaud Lacombe wrote:
>>>> Hi,
>>>>
>>>> On Wed, Sep 14, 2011 at 10:53 PM, Arnaud Lacombe
>>>> wrote:
>>>>> Hi Kip,
>>>>>
>>>>> I've got a few question about the buf_ring(9) API.
>>>>>
>>>>> 1) what means the 'drbr_' prefix. I can guess the two last letter, 'b'
>>>>> and 'r', for Buffer Ring, but what about 'd' and 'r' ?
>>>>>
>>>>> 2) in `sys/sys/buf_ring.h', you defined 'struct buf_ring' as:
>>>>>
>>>>> struct buf_ring {
>>>>> volatile uint32_t br_prod_head;
>>>>> volatile uint32_t br_prod_tail;
>>>>> int br_prod_size;
>>>>> int br_prod_mask;
>>>>> uint64_t br_drops;
>>>>> uint64_t br_prod_bufs;
>>>>> uint64_t br_prod_bytes;
>>>> shouldn't those 3 fields be updated atomically, especially on 32bits
>>>> platforms ? That might pose a problem as, AFAIK, FreeBSD do not have
>>>> MI 64bits atomics operations...
>>>
>>> Between the point at which br_prod_tail == prod_head and when we
>>> update br_prod_tail to point to prod_next we are the exclusive owners
>>> of the fields in buf_ring. That is why we wait for any other
>>> enqueueing threads to update br_prod_tail to point to prod_head before
>>> continuing.
>>>
>> How do you enforce ordering ? I do not see anything particular
>> forbidding the `br->br_prod_tail' to be committed first, leading other
>> thread to believe they have access to the statistics, while the other
>> thread has not yet committed its change.
>>
>> Thanks,
>> - Arnaud
>>
>>> Cheers
>>>
>>> /*
>>> * If there are other enqueues in progress
>>> * that preceeded us, we need to wait for them
>>> * to complete
>>> */
>>> while (br->br_prod_tail != prod_head)
>>> cpu_spinwait();
>>> br->br_prod_bufs++;
>>> br->br_prod_bytes += nbytes;
>>> br->br_prod_tail = prod_next;
>>> critical_exit();
>>>
>>
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein wrote: > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >> >> Just decided to follow the global trends and trying to throw all of my >> local/private changes at you in hope that the "crowd-sourcing magic" might >> somehow happen :-) This seems definitely easier than carefully producing the >> patch files and keeping them up-to-date. >> >> So, my newly cloned gitorious repository: >> https://gitorious.org/~avg/freebsd/avgbsd >> >> And the first branch of interest: >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 > > I'll throw mine in as well: > > https://github.com/uqs/freebsd-head/branches > is that the same as tree as Fabien's, or a new snapshot ? If not, couldn't we agree to have all the same tree in order to ease code sharing between all of them ? I see there is already a https://github.com/freebsd/freebsd tree, which seem different than Fabien's tree on github. It really look like The FreeBSD Project (https://github.com/freebsd/) is not able to provide consistency. Thanks, - Arnaud > If you did the rsync from repos.freebsd.your.org, then you simply: > > $ git remote add uqs git://github.com/uqs/freebsd-head.git > $ git fetch uqs > > and you can look at the remote branches uqs/* to see bits of it. > > Cheers, > Uli > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Thu, Sep 22, 2011 at 5:26 PM, Ulrich Spörlein wrote: > On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein wrote: >> > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >> >> >> >> Just decided to follow the global trends and trying to throw all of my >> >> local/private changes at you in hope that the "crowd-sourcing magic" might >> >> somehow happen :-) This seems definitely easier than carefully producing >> >> the >> >> patch files and keeping them up-to-date. >> >> >> >> So, my newly cloned gitorious repository: >> >> https://gitorious.org/~avg/freebsd/avgbsd >> >> >> >> And the first branch of interest: >> >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 >> > >> > I'll throw mine in as well: >> > >> > https://github.com/uqs/freebsd-head/branches >> > >> is that the same as tree as Fabien's, or a new snapshot ? >> >> If not, couldn't we agree to have all the same tree in order to ease >> code sharing between all of them ? >> >> I see there is already a https://github.com/freebsd/freebsd tree, >> which seem different than Fabien's tree on github. It really look like >> The FreeBSD Project (https://github.com/freebsd/) is not able to >> provide consistency. > > The freebsd-head tree is the same that everybody in the world can get by > simply running git svn clone against the FreeBSD subversion server > (you'd need a lot of patience, though). > > It's also available from git.freebsd.your.org and on code.google.com > (except that Google's git backend sucks ass). > > Fabien's tree is cut off at an arbitrary date and if you happen to > choose a different date, well you cannot merge any branches with that > repository. > ok, so Fabien's tree is the one duplicating the work :) looks like for years I used the wrong one and better have to switch to the tree from github to get the full history. Fabien, if the size is the only matter of your tree, shouldn't you just create a shallow tree from the common tree ? At least, hashes will match up to a given point. You may also want to clean the branches to only keep releng/, stable/, releases/ and head/. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Thu, Sep 22, 2011 at 5:26 PM, Ulrich Spörlein wrote: > On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein wrote: >> > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >> >> >> >> Just decided to follow the global trends and trying to throw all of my >> >> local/private changes at you in hope that the "crowd-sourcing magic" might >> >> somehow happen :-) This seems definitely easier than carefully producing >> >> the >> >> patch files and keeping them up-to-date. >> >> >> >> So, my newly cloned gitorious repository: >> >> https://gitorious.org/~avg/freebsd/avgbsd >> >> >> >> And the first branch of interest: >> >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 >> > >> > I'll throw mine in as well: >> > >> > https://github.com/uqs/freebsd-head/branches >> > >> is that the same as tree as Fabien's, or a new snapshot ? >> >> If not, couldn't we agree to have all the same tree in order to ease >> code sharing between all of them ? >> >> I see there is already a https://github.com/freebsd/freebsd tree, >> which seem different than Fabien's tree on github. It really look like >> The FreeBSD Project (https://github.com/freebsd/) is not able to >> provide consistency. > > The freebsd-head tree is the same that everybody in the world can get by > simply running git svn clone against the FreeBSD subversion server > (you'd need a lot of patience, though). > FWIW, how comes that there is not yet any `stable/9' branch on the github tree ? Thanks, - Arnaud > It's also available from git.freebsd.your.org and on code.google.com > (except that Google's git backend sucks ass). > > Fabien's tree is cut off at an arbitrary date and if you happen to > choose a different date, well you cannot merge any branches with that > repository. > > http://wiki.freebsd.org/GitWorkflow > > Uli > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Installation of kernel symbols file in a separate directory [Was: Re: Experiences with FreeBSD 9.0-BETA2]
Hi,
[Moving the discussion on -hackers@]
On Tue, Sep 27, 2011 at 6:02 PM, Arnaud Lacombe wrote:
> Hi,
>
> On Tue, Sep 27, 2011 at 5:01 PM, Peter Jeremy wrote:
>> On 2011-Sep-26 19:48:23 -0400, Benjamin Kaduk wrote:
>>>On Mon, 26 Sep 2011, Arnaud Lacombe wrote:
>>>> The problem with /boot on a dedicated partition is the the kernel,
>>>> since at least 8.x, is installed by default with a vast majority of
>>>> crap. That's all the .symbols, that 99% of FreeBSD users will never
>>>> uses.
>>>
>>>My recollection is that this is because kensmith forgot to take
>>>'makeoptions DEBUG=-g' out of GENERIC when branching stable/8, and no one
>>
>> Not quite - 'DEBUG=-g' was a deliberate move to make it easier for
>> developers to talk users through faultfinding kernel issues.
>>
>> The correct fix is to install the .symbols files somewhere other than
>> /boot/kernel - unfortunately, no-one has developed the necessary
>> changes to the kernel installation.
>>
> I did too, will put patches online soon.
>
available here:
https://github.com/lacombar/freebsd/commit/c1d5c4857361028dc40941273dd2ab9576c687fd
This patch adds a KODIR_SYMBOLS variables which is to the kernel and
modules .symbols files what the KODIR variable is to the kernel and
modules files. By default ${KODIR_SYMBOLS} = ${KODIR}.
- Arnaud
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: my git development snapshot(s)
Hi, On Sun, Sep 18, 2011 at 5:14 AM, Andriy Gapon wrote: > > Just decided to follow the global trends and trying to throw all of my > local/private changes at you in hope that the "crowd-sourcing magic" might > somehow happen :-) This seems definitely easier than carefully producing the > patch files and keeping them up-to-date. > > So, my newly cloned gitorious repository: > https://gitorious.org/~avg/freebsd/avgbsd > and mine: https://github.com/lacombar/freebsd/branches not much stuff for now, but should fill up in the next few days. This a fork from the freebsd/freebsd.git tree on github, with only stable branches in it (ie. none of the >2k branches of the original clone), plus my stuff. - Arnaud > And the first branch of interest: > https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 > > This is a snapshot of almost all of my local changes to the FreeBSD src tree. > Please note that the branch is not intended to be public in the git sense. > That > is, there will be no linear history - I periodically rebase my changes on top > of > the svn head and also frequently reshuffle/merge/split commits. > The snapshot is not tidied up, there are quite a few commits that should be > merged into other commits, some commit messages are not accurate/pretty, etc. > The older the commits, the more mature they are supposed to be. > > Based on the above, no new commits are expected to this snapshot branch. > I will produce new snapshot branches from time to time. > > I am posting this information to this list initially, later I plan to share > the > code with the wider audience e.g. via hackers@. > > P.S. This code sharing is made easier for me by git, Gitorious and "git rebase > --onto" in particular. Thanks to Fabien Thomas for the initial FreeBSD clone > repository at Gitorious! > > -- > Andriy Gapon > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Installation of kernel symbols file in a separate directory [Was: Re: Experiences with FreeBSD 9.0-BETA2]
Hi,
On Wed, Sep 28, 2011 at 8:29 AM, Andriy Gapon wrote:
> on 28/09/2011 07:23 Arnaud Lacombe said the following:
>> available here:
>>
>> https://github.com/lacombar/freebsd/commit/c1d5c4857361028dc40941273dd2ab9576c687fd
>>
>> This patch adds a KODIR_SYMBOLS variables which is to the kernel and
>> modules .symbols files what the KODIR variable is to the kernel and
>> modules files. By default ${KODIR_SYMBOLS} = ${KODIR}.
>
> I think that this patch doesn't cover the case of doing make install in a
> module
> directory (i.e. module built/installed independently from a kernel).
> KMODDIR_SYMBOLS should have a default value like KMODDIR does (in bsd.own.mk).
>
that's trivial to fix. As I won't commit that anyway, and the patch
will not be taken as-is, feel free to make the change. Still, I may
end-up doing it by myself when I'll need it.
- Arnaud
> --
> Andriy Gapon
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe wrote: > Hi, > > On Thu, Sep 22, 2011 at 5:26 PM, Ulrich Spörlein wrote: >> On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: >>> Hi, >>> >>> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein wrote: >>> > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >>> >> >>> >> Just decided to follow the global trends and trying to throw all of my >>> >> local/private changes at you in hope that the "crowd-sourcing magic" >>> >> might >>> >> somehow happen :-) This seems definitely easier than carefully >>> >> producing the >>> >> patch files and keeping them up-to-date. >>> >> >>> >> So, my newly cloned gitorious repository: >>> >> https://gitorious.org/~avg/freebsd/avgbsd >>> >> >>> >> And the first branch of interest: >>> >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 >>> > >>> > I'll throw mine in as well: >>> > >>> > https://github.com/uqs/freebsd-head/branches >>> > >>> is that the same as tree as Fabien's, or a new snapshot ? >>> >>> If not, couldn't we agree to have all the same tree in order to ease >>> code sharing between all of them ? >>> >>> I see there is already a https://github.com/freebsd/freebsd tree, >>> which seem different than Fabien's tree on github. It really look like >>> The FreeBSD Project (https://github.com/freebsd/) is not able to >>> provide consistency. >> >> The freebsd-head tree is the same that everybody in the world can get by >> simply running git svn clone against the FreeBSD subversion server >> (you'd need a lot of patience, though). >> > FWIW, how comes that there is not yet any `stable/9' branch on the github > tree ? > Ulrich, ping ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NUMA Support is there in FreeBSD.
Hi, On Mon, Oct 3, 2011 at 12:31 PM, wrote: > On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli > wrote: >> I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. >> If FreeBSD supports NUMA what are the kernel API to allocate memory? >> is there any example driver or any driver which is using the NUMA API? >> >> please provide some inputs... > > The kernel is NUMA-aware (at least for x86), > What "x86" ? i386 ? amd64 ? both ? > and memory is allocated > round-robin amongst the memory domains. There are not yet any KPIs > for allocating memory in a specific NUMA domain, nor for binding > specific threads / processes to get their memory local to a bound cpu > instead of round robin. > I'm not sure to follow you. Say you have 2 memory domain attached to 2 different CPU package, each providing a memory domain, 4 physical core and eventually 8 virtual. Say you have a network adapter supporting 8 RX/TX queue, dispatching RX packet to 8 netisr. Ideally, you'd want those 8 queue/netisr to each have an affinity for a given CPU/memory domain, have the network adapter route flow evenly on those those 8 CPU. Now, if you allocated an mbuf from memory domain 1, and end up being processed by a CPU in domain 0, that likely to introduce performance penalty. Now, what about userland ? This is certainly an horribly big picture :/ - Arnaud > There have been several discussions but no one has taken the lead and > proposed some KPIs yet. At $WORK the round-robin is sufficient to get > consistent performance numbers and we have not yet started any > experimentation with binding specific threads to either CPU or memory. > > Cheers, > matthew > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Installation of kernel symbols file in a separate directory [Was: Re: Experiences with FreeBSD 9.0-BETA2]
Hi, On Wed, Sep 28, 2011 at 10:30 AM, Ryan Stone wrote: > You might be interested in this PR: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=153157&cat= > > This does the same thing for userland .symbols files, and teaches gdb > how to find them. > Actually, I've made my own implementation of this. It is available here: https://github.com/lacombar/freebsd/commit/e6cb6f2d20ffb6a060124f98c4f4cc660fe6f8bb However, it is a bit more intrusive, as highlighted in the following: https://github.com/lacombar/freebsd/commit/8850b8acf5008c6f9de019ebc3691b090c894605 - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: NUMA Support is there in FreeBSD.
Hi, [Add jhb@ to the CC list] On Mon, Oct 3, 2011 at 1:34 PM, wrote: > On Mon, Oct 3, 2011 at 10:24 AM, Arnaud Lacombe wrote: >> Hi, >> >> On Mon, Oct 3, 2011 at 12:31 PM, wrote: >>> On Mon, Oct 3, 2011 at 7:55 AM, satish kondapalli >>> wrote: >>>> I am new to FreeBSD, I just want know whether FreeBSD supports NUMA. >>>> If FreeBSD supports NUMA what are the kernel API to allocate memory? >>>> is there any example driver or any driver which is using the NUMA API? >>>> >>>> please provide some inputs... >>> >>> The kernel is NUMA-aware (at least for x86), >>> >> What "x86" ? i386 ? amd64 ? both ? > > Both; see sys/x86/acpica/srat.c which parses the SRAT table. > >>> and memory is allocated >>> round-robin amongst the memory domains. There are not yet any KPIs >>> for allocating memory in a specific NUMA domain, nor for binding >>> specific threads / processes to get their memory local to a bound cpu >>> instead of round robin. >>> >> I'm not sure to follow you. Say you have 2 memory domain attached to 2 >> different CPU package, each providing a memory domain, 4 physical core >> and eventually 8 virtual. Say you have a network adapter supporting 8 >> RX/TX queue, dispatching RX packet to 8 netisr. Ideally, you'd want >> those 8 queue/netisr to each have an affinity for a given CPU/memory >> domain, have the network adapter route flow evenly on those those 8 >> CPU. Now, if you allocated an mbuf from memory domain 1, and end up >> being processed by a CPU in domain 0, that likely to introduce >> performance penalty. > > Your statement isn't incorrect. What I'm saying is that there's no > KPI for requesting bound memory because, while the netstat example is > a fine one for where local memory is desired, the majority [1] of > processing is not bound to a CPU and so round-robin allocations will > produce uniform performance results -- that is, not the best possible, > but not wildly fluctuating as scheduling decisions over different runs > give different remote memory penalties. > > [1] for some definition of 'majority'. > >> Now, what about userland ? >> >> This is certainly an horribly big picture :/ > > Yes, and it's why I said just that there's no KPI. One reason there > is no KPI is that there's a lot of fiddly bits to take into account. > > My experience at IBM on AIX was that NUMA is very easy to get wrong; > specifically what one usually wants is for the OS to get the answer > right (especially for userspace) without a lot of manual tuning; > except for some specific applications like netstat queues or a machine > doing HPC or mostly running e.g. an Oracle db server, there's too much > happening for any one program to configure itself "right" for all the > uses of that code. I remember a lot of customer reports of problems > from overly aggressive local memory use. Most of the time no one > complained when things had consistent performance, even if that wasn't > quite as fast as possible. > Is there any project in progress to get this addressed ? In the past year, I can only see 3 commit related to NUMA, one of them being concerning only ia64. Btw, I'd be interested to see how FreeBSD 9.0 and a recent Linux kernel behave on +2 CPU package machines. - Arnaud [0]: http://lwn.net/Articles/254445/ > In fact, I may be wrong about the round-robin; I sent jhb@ a patch and > I have no recollection anymore whether it's actually in CURRENT. It's > been over a year since I thought about this much (BSDCan 2010 was the > last time I remember). > > Cheers, > matthew > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, On Thu, Nov 3, 2011 at 9:58 AM, Aleksandr Rybalko wrote: > On Thu, 3 Nov 2011 10:22:22 +0100 > Lars Engels wrote: > >>> Hi Hackers, >>> >>> maybe you've heard of the upcoming Raspberry Pi, a credit card sized >>> ARM computer which is about to get sold for 25$ - 35$ from december >>> on. >>> >>> http://www.raspberrypi.org/archives/302 >>> >>> Hardware details: http://elinux.org/RaspberryPiBoard#Hardware_Details >>> >>> The first charge will be 10,000 pieces, so there's a fair chance that >>> one can actually buy a board. >>> >>> If someone is willing to port FreeBSD to the Raspberry, I'd try to >>> get one of the boards and send it to the porter. >>> >>> Cheers >>> Lars > > Hi Lars, > > I seen info about Raspberry Pi, this is very nice hardware. > Are you kidding ? ARM11 is a 10 year old core, not much to be really proud with. The only interesting thing about the Pi is its price tag, but as you point out, if we don't have datasheets upfront, it's not going to be much than a $25 paper-weight. Btw, do FreeBSD provide any KPI for hardware video "accelerator" ? AFAIK, we do not even have such framework for crypto accelerator, so I serious doubt there is any for video :/ - Arnaud > It will be > fun to port FreeBSD on it, since I already have experience with > Broadcom SoC's. But device have undocumented parts, so writing driver > for it maybe impossible or very hard (through a lot of RE). > > Anyway, it would be nice to play with it. > > WBW > -- > Aleksandr Rybalko > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, [Please do not top-post.] On Thu, Nov 3, 2011 at 11:20 AM, Nate Dobbs wrote: > 10 year old core or not, the ARM is the worlds most widely used processor; > Please read what I said correctly, I said "this ARM11 is obsolete" (even if still used, for sure) and "FreeBSD lacks critical KPI". Concerning the first point; you want a nice ARM core ? take the PandaBoard, CortexA9, dual-core, DDR2. However, porting FreeBSD on this chip will be way more difficult, and nobody will do the work is the next year or two (my guess). Concerning the second point, Unix may be 43 years old, you surely do not want to run 4.3BSD on the latest NUMA system, and expect optimum platform usage. Someone has to make the damn code do the job. Even as of FreeBSD 9.0, we won't even have complete NUMA support. I'm taking here about a *mainstream* system architecture, not even a deeply embedded and integrated system. - Arnaud > On Thu, Nov 3, 2011 at 11:12 AM, Arnaud Lacombe wrote: >> >> Hi, >> >> On Thu, Nov 3, 2011 at 9:58 AM, Aleksandr Rybalko wrote: >> > On Thu, 3 Nov 2011 10:22:22 +0100 >> > Lars Engels wrote: >> > >> >>> Hi Hackers, >> >>> >> >>> maybe you've heard of the upcoming Raspberry Pi, a credit card sized >> >>> ARM computer which is about to get sold for 25$ - 35$ from december >> >>> on. >> >>> >> >>> http://www.raspberrypi.org/archives/302 >> >>> >> >>> Hardware details: http://elinux.org/RaspberryPiBoard#Hardware_Details >> >>> >> >>> The first charge will be 10,000 pieces, so there's a fair chance that >> >>> one can actually buy a board. >> >>> >> >>> If someone is willing to port FreeBSD to the Raspberry, I'd try to >> >>> get one of the boards and send it to the porter. >> >>> >> >>> Cheers >> >>> Lars >> > >> > Hi Lars, >> > >> > I seen info about Raspberry Pi, this is very nice hardware. >> > >> Are you kidding ? >> >> ARM11 is a 10 year old core, not much to be really proud with. The >> only interesting thing about the Pi is its price tag, but as you point >> out, if we don't have datasheets upfront, it's not going to be much >> than a $25 paper-weight. >> >> Btw, do FreeBSD provide any KPI for hardware video "accelerator" ? >> AFAIK, we do not even have such framework for crypto accelerator, so I >> serious doubt there is any for video :/ >> >> - Arnaud >> >> > It will be >> > fun to port FreeBSD on it, since I already have experience with >> > Broadcom SoC's. But device have undocumented parts, so writing driver >> > for it maybe impossible or very hard (through a lot of RE). >> > >> > Anyway, it would be nice to play with it. >> > >> > WBW >> > -- >> > Aleksandr Rybalko >> > ___ >> > freebsd-hackers@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> > To unsubscribe, send any mail to >> > "freebsd-hackers-unsubscr...@freebsd.org" >> > >> ___ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" > > > > -- > Cheers, > > Nate Dobbs RHCE > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, [added gber@ and cognet@ to the Cc: list] On Thu, Nov 3, 2011 at 9:19 AM, richo wrote: > On 03/11/11 10:22 +0100, Lars Engels wrote: >> >> Hi Hackers, >> >> maybe you've heard of the upcoming Raspberry Pi, a credit card sized ARM >> computer which is about to get sold for 25$ - 35$ from december on. >> >> http://www.raspberrypi.org/archives/302 >> >> Hardware details: http://elinux.org/RaspberryPiBoard#Hardware_Details >> >> The first charge will be 10,000 pieces, so there's a fair chance that >> one can actually buy a board. >> >> If someone is willing to port FreeBSD to the Raspberry, I'd try to get >> one of the boards and send it to the porter. >> >> Cheers >> Lars > > I am planning to attempt a port, however I'm very new to kernel > development/FreeBSD and would not feel right about accepting hardware for a > project I'm not sure I have the skills to finish. > > To that end I'll be buying boards on release date, but would be interested > in working with anyone else who is attempting it. > You certainly want to get in touch with g...@freebsd.org and cog...@freebsd.org, as well as getting familiar with the code in `sys/arm/', especially the one in the `projects/armv6' branch in the p4 repository and the various TRM you could find on the ARM11 core from ARM Ltd. gber@, cognet@, what is the status of the code in the p4 repository ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, On Thu, Nov 3, 2011 at 8:40 PM, Greg 'groggy' Lehey wrote: > On Thursday, 3 November 2011 at 11:33:25 -0400, Arnaud Lacombe wrote: >> [Please do not top-post.] > > Please trim messages. > >> On Thu, Nov 3, 2011 at 11:20 AM, Nate Dobbs >> wrote: >>> 10 year old core or not, the ARM is the worlds most widely used processor; >>> >> Please read what I said correctly, I said "this ARM11 is obsolete" >> (even if still used, for sure) ... > > Clearly price is an issue for this device. What's so bad about ARM11 > that it shouldn't be used? > If you read my original comment, I did point out the $25 price tag was pretty much the only interesting thing. Now, what it has been designed for, multimedia, is going to be handled by a closed-source binary blob without datasheet, so let me turn back the question: what do you expect doing with it ? >From my point of view, I would be more interested into bringing up FreeBSD on ARMv7 (ie. Cortex A[89]), rather than any previous, but the effort is clearly not the same. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, On Thu, Nov 3, 2011 at 10:05 PM, Nate Dobbs wrote: > On Thu, Nov 3, 2011 at 9:55 PM, Greg 'groggy' Lehey > wrote: >> >> On Thursday, 3 November 2011 at 21:05:54 -0400, Arnaud Lacombe wrote: >> > On Thu, Nov 3, 2011 at 8:40 PM, Greg 'groggy' Lehey >> > wrote: >> >> On Thursday, 3 November 2011 at 11:33:25 -0400, Arnaud Lacombe wrote: >> >>> On Thu, Nov 3, 2011 at 11:20 AM, Nate Dobbs >> >>> wrote: >> >>>> 10 year old core or not, the ARM is the worlds most widely used >> >>>> processor; >> >>>> >> >>> Please read what I said correctly, I said "this ARM11 is obsolete" >> >>> (even if still used, for sure) ... >> >> >> >> Clearly price is an issue for this device. What's so bad about ARM11 >> >> that it shouldn't be used? >> >> >> > If you read my original comment, I did point out the $25 price tag was >> > pretty much the only interesting thing. Now, what it has been designed >> > for, multimedia, is going to be handled by a closed-source binary blob >> > without datasheet, so let me turn back the question: what do you >> > expect doing with it ? >> >> That's not turning back the question; that's a separate question. But >> it's a good one. I don't really see it as a multimedia device. My >> interest would be in little embedded agents in different parts of the >> house, for things like measuring temperatures. I'm sure lots of other >> applications will come to mind. >> >> And yes, I'll probably use the supplied Linux port. But if a FreeBSD >> alternative becomes available, I'd certainly prefer that. >> >> Greg >> -- >> Sent from my desktop computer >> Finger g...@freebsd.org for PGP public key. >> See complete headers for address and phone numbers. >> This message is digitally signed. If your Microsoft MUA reports >> problems, please read http://tinyurl.com/broken-mua > > I agree with groggy, something I'd personally use it for is a small SSH > server to allow a pinhole into my home network. It would serve as a very > good replacement for the mac mini that's sitting in my DMZ simply handling > connections for my SSH tunnel so I can bypass the proxy at work. > > Power savings would be significant and it would be plenty powerful to handle > this task. A small webcam server comes to mind as well; there could be > plenty of useful things I could think of outside the realm of multimedia. > you certainly want: http://beagleboard.org/bone $89, 700MHz Cortex A8, 256MB DRR2, micro-SD. However, do not expect being able to run FreeBSD on it before a few years :) - Arnaud > JMHO > > > > -- > Cheers, > > Nate Dobbs RHCE > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
FreeBSD on ARM Cortex board [Was: Porting FreeBSD to Raspberry Pi]
Hi, [Starting a new thread, added Ben Gray to the Cc: list] On Thu, Nov 3, 2011 at 10:30 PM, Arnaud Lacombe wrote: > Hi, > > On Thu, Nov 3, 2011 at 10:05 PM, Nate Dobbs > wrote: >> On Thu, Nov 3, 2011 at 9:55 PM, Greg 'groggy' Lehey >> wrote: >>> >>> On Thursday, 3 November 2011 at 21:05:54 -0400, Arnaud Lacombe wrote: >>> > On Thu, Nov 3, 2011 at 8:40 PM, Greg 'groggy' Lehey >>> > wrote: >>> >> On Thursday, 3 November 2011 at 11:33:25 -0400, Arnaud Lacombe wrote: >>> >>> On Thu, Nov 3, 2011 at 11:20 AM, Nate Dobbs >>> >>> wrote: >>> >>>> 10 year old core or not, the ARM is the worlds most widely used >>> >>>> processor; >>> >>>> >>> >>> Please read what I said correctly, I said "this ARM11 is obsolete" >>> >>> (even if still used, for sure) ... >>> >> >>> >> Clearly price is an issue for this device. What's so bad about ARM11 >>> >> that it shouldn't be used? >>> >> >>> > If you read my original comment, I did point out the $25 price tag was >>> > pretty much the only interesting thing. Now, what it has been designed >>> > for, multimedia, is going to be handled by a closed-source binary blob >>> > without datasheet, so let me turn back the question: what do you >>> > expect doing with it ? >>> >>> That's not turning back the question; that's a separate question. But >>> it's a good one. I don't really see it as a multimedia device. My >>> interest would be in little embedded agents in different parts of the >>> house, for things like measuring temperatures. I'm sure lots of other >>> applications will come to mind. >>> >>> And yes, I'll probably use the supplied Linux port. But if a FreeBSD >>> alternative becomes available, I'd certainly prefer that. >>> >>> Greg >>> -- >>> Sent from my desktop computer >>> Finger g...@freebsd.org for PGP public key. >>> See complete headers for address and phone numbers. >>> This message is digitally signed. If your Microsoft MUA reports >>> problems, please read http://tinyurl.com/broken-mua >> >> I agree with groggy, something I'd personally use it for is a small SSH >> server to allow a pinhole into my home network. It would serve as a very >> good replacement for the mac mini that's sitting in my DMZ simply handling >> connections for my SSH tunnel so I can bypass the proxy at work. >> >> Power savings would be significant and it would be plenty powerful to handle >> this task. A small webcam server comes to mind as well; there could be >> plenty of useful things I could think of outside the realm of multimedia. >> > you certainly want: > > http://beagleboard.org/bone > > $89, 700MHz Cortex A8, 256MB DRR2, micro-SD. However, do not expect > being able to run FreeBSD on it before a few years :) > actually, some initial work has been started by Ben Gray: http://code.google.com/p/beagleboard-freebsd/ and https://gitorious.org/+freebsd-omap-team/freebsd/freebsd-omap/ - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, 2011/11/4 Lev Serebryakov : > Hello, Arnaud. > You wrote 4 ноября 2011 г., 6:30:18: > >> $89, 700MHz Cortex A8, 256MB DRR2, micro-SD. However, do not expect >> being able to run FreeBSD on it before a few years :) > What is so special about A8? We have some ARM code in tree, is Cortex > SO different from previous architectures? As far as I understand, main > problem in such platforms is different perefireal devices on each next > chop and board, which doesn't have "common denominator" like PCI bus > or other standard pieces, which helps a lot on PC (x86, amd64) architecture. > > BTW, we have infrastructure for crypto accelerators ;-) > Yes, my bad. > And what about NUMA... I know only one (almost-general-purpose) OS, > which has full support for NUMA. It is Sun^WOracle Solaris. Ant it > does it very well (not to forget about boxes like SunFire 25K with 72 > 2-core sockets at, as fa as I remember, 18 processor-memory boards). > > I participated in project, which added NUMA support to Sun JVM > HotSpot, and Solaris was only one which gives us all needed > information -- nor Linux nor Windows either had not needed API two > years ago, and as far as I know, there is no big changes in this area > for them (but I could be wrong). > FWIW, referring to the technology two years ago in computer science is just like referring to the Prehistory to make a point about mankind today :/ - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, On Fri, Nov 4, 2011 at 10:57 AM, Adrian Chadd wrote: > On 4 November 2011 07:53, Arnaud Lacombe wrote: > >> FWIW, referring to the technology two years ago in computer science is >> just like referring to the Prehistory to make a point about mankind >> today :/ > > .. and that's actually a valid thing to do. Not everything old is: > > * irrelevant; > * not applicable; > * useless to know. > > I think something high profile like the Pi would be great to support. > I heard the same thing about NetBSD and OpenMoko 3 or 4 years ago, nothing came through and it was just a rather standard ARM926. > So, let's do it. ;) > I still haven't heard from either gber@, cognet@ about the current status of `projects/armv6', especially wrt. known issues. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Porting FreeBSD to Raspberry Pi
Hi, 2011/11/4 Lev Serebryakov : > Hello, Arnaud. > You wrote 4 ноября 2011 г., 6:30:18: > >> $89, 700MHz Cortex A8, 256MB DRR2, micro-SD. However, do not expect >> being able to run FreeBSD on it before a few years :) > What is so special about A8? > It is the consumer technology of today. The best people can afford without being in ARM's R&D centers. Let me tell you what is going to happen. ARM11 has been around for years, it will take you a year or two to complete the project, nice, hacker thrill, you did it. However, by the time you release it, the Raspberry Pi will be sold-out and will be replaced by an ARMv7 core, smaller, faster, eventually cheaper. By that time, the current technology will be a 64bits MP-core ARMv8, And you will be in the exact same situation as today, FreeBSD lagging one or two generation behind Linux, keep up. Now, I agree, no battle is lost if there one fool to fight for it. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein wrote: > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe wrote: >> > Hi, >> > >> > On Thu, Sep 22, 2011 at 5:26 PM, Ulrich Spörlein >> > wrote: >> >> On Thu, 2011-09-22 at 15:52:43 -0400, Arnaud Lacombe wrote: >> >>> Hi, >> >>> >> >>> On Thu, Sep 22, 2011 at 3:05 PM, Ulrich Spörlein >> >>> wrote: >> >>> > On Sun, 2011-09-18 at 12:14:38 +0300, Andriy Gapon wrote: >> >>> >> >> >>> >> Just decided to follow the global trends and trying to throw all of my >> >>> >> local/private changes at you in hope that the "crowd-sourcing magic" >> >>> >> might >> >>> >> somehow happen :-) This seems definitely easier than carefully >> >>> >> producing the >> >>> >> patch files and keeping them up-to-date. >> >>> >> >> >>> >> So, my newly cloned gitorious repository: >> >>> >> https://gitorious.org/~avg/freebsd/avgbsd >> >>> >> >> >>> >> And the first branch of interest: >> >>> >> https://gitorious.org/~avg/freebsd/avgbsd/commits/devel-20110915 >> >>> > >> >>> > I'll throw mine in as well: >> >>> > >> >>> > https://github.com/uqs/freebsd-head/branches >> >>> > >> >>> is that the same as tree as Fabien's, or a new snapshot ? >> >>> >> >>> If not, couldn't we agree to have all the same tree in order to ease >> >>> code sharing between all of them ? >> >>> >> >>> I see there is already a https://github.com/freebsd/freebsd tree, >> >>> which seem different than Fabien's tree on github. It really look like >> >>> The FreeBSD Project (https://github.com/freebsd/) is not able to >> >>> provide consistency. >> >> >> >> The freebsd-head tree is the same that everybody in the world can get by >> >> simply running git svn clone against the FreeBSD subversion server >> >> (you'd need a lot of patience, though). >> >> >> > FWIW, how comes that there is not yet any `stable/9' branch on the github >> > tree ? >> > >> Ulrich, ping ? > > Oops, sorry for the delay! Fixed now, thanks for bringing it to my > attention. I missed the push --all flag. :/ > FWIW, the github mirror[0] of the completed SVN tree has not seen any upgrade for a week now. Did some script broke ? Thanks, - Arnaud [0]: https://github.com/freebsd/freebsd ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
Hi, On Sat, Nov 19, 2011 at 5:01 AM, Gleb Kurtsou wrote: > Hi, > > I was lucky to write a bit of code which gcc 4.2 fails to compile > correctly with -O2. Too keep long story short the code fails for gcc > from base system and last gcc 4.2 snapshot from ports. It works with gcc > 4.3, gcc 4.4 on FreeBSD and Linux. Clang from base is also good. -O and > -Os optimization levels are fine (I've tried with all -f* flags > mentioned in documentation) > > -O2 -fno-omit-frame-pointer combination is troublesome on amd64. I > presume i386 should be fine. These options are also used for > compilation of kernel (with debugging enabled) and modules. > > I'm not able to share the code, but have a test case reproducing the > bug. I've encountered the issue over a week ago and tried narrowing it down > to a simple test I could share but without much success. > > The code itself is very common: initialize two structs on stack, call a > function with pointers to those stucts as arguments. A number of inlined > assertion functions. gcc fails to correctly optimize struct assignments > with -fno-omit-frame-pointer, I have a number of small structs assigned, > gcc decides not to use data coping but to assign fields directly. I've > tried disabling sra, tweaking sra parameters -- no luck in forcing it > to copy data. Replacing one particular assignment with memcpy produces > correct code, but that's not a solution. > > -O2 -fno-omit-frame-pointer -fno-inline is buggy > -O2 -fno-omit-frame-pointer -frename-registers is buggy > > I found similar issue with gcc 4.6, but I'm not able to reproduce it > with gcc test case: > https://bugzilla.redhat.com/show_bug.cgi?id=679924 > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47893 > this PR seems highly irrelevant, the cause has been identified to a commit made in mid-2010, that's 3 years older than gcc in base. > I'll be glad to help debugging it and will be hanging on #bsddev during > weekend as glk. > at least, can you share the testcase and miscompilation details ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: gcc 4.2 miscompilation with -O2 -fno-omit-frame-pointer on amd64
Hi, On Fri, Dec 9, 2011 at 10:15 AM, Rafal Jaworowski wrote: > > On 2011-12-08, at 17:53, Nathan Whitehorn wrote: > >> On 12/08/11 03:01, Piotr Nowak wrote: >>> We're working on PowerPC target using GCC 4.2.1 >>> and FreeBSD 6.1. It seems like we have similar >>> problem. In our case GCC sometimes very unfortunately >>> optimize code with -fno-omit-frame-pointer. >>> >>> Example shown below covers file sys/powerc/booke/pmap.c >>> and function pmap_kenter. If we disassemble kernel binary >>> we have: >>> >>> c019998c: 4b ec 6a ed bl c0060478<_mtx_unlock_spin_flags> >>> c010: 81 61 00 00 lwz r11,0(r1) >>> c014: 80 0b 00 04 lwz r0,4(r11) >>> c018: 7d 61 5b 78 mr r1,r11 >>> c01c: 82 ab ff d4 lwz r21,-44(r11) >>> c01999a0: 7c 08 03 a6 mtlr r0 >>> c01999a4: 82 cb ff d8 lwz r22,-40(r11) >>> c01999a8: 82 eb ff dc lwz r23,-36(r11) >>> c01999ac: 83 0b ff e0 lwz r24,-32(r11) >>> c01999b0: 83 2b ff e4 lwz r25,-28(r11) >>> c01999b4: 83 4b ff e8 lwz r26,-24(r11) >>> c01999b8: 83 6b ff ec lwz r27,-20(r11) >>> >>> As you can see stack pointer on R1 is being updated >>> before stashed data were pulled off stack. (mr r1,r11) >>> As a result of this we have chance to get crash when >>> any interrupt hit shortly after stack pointer update. >>> The interrupt prologue will override not yet pulled off >>> pmap_kenter function data. >>> >>> The problem occures only with -fno-omit-frame-pointer >>> and not every branch returns are beeing corrupted. >>> >>> Do you think this issue may be somehow related to yours? >>> Are there any patches/solutions to fix it? >> >> Should we turn off -fno-omit-frame-frame-pointer on PPC then? It's enabled >> in default kernel builds. > > I think that's a good idea. Even though we have managed to trigger this only > in rare cases, the problem is real and the code generated is broken i.e. > leads to corruption and panics. > How can you make any conclusion without having seen a single line of code actually triggering the problem ? That sounds very irresponsible... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: loop inside uma_zfree critical section
Hi, On Wed, Dec 14, 2011 at 2:47 PM, Monthadar Al Jaberi wrote: > On Tue, Dec 13, 2011 at 4:50 PM, Monthadar Al Jaberi > wrote: >> On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: >>> On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: Hi, I am not sure why I am having this problem, but looking at the code I dont understand uma_core.c really good. So I hope someone can shed a light on this: I am running on an arm board and and running a kernel module that behaves like a wlan interface. so I tx and rx packets. For now tx is only only sending beacon like frames. This is done through using ieee80211_beacon_alloc(). Then in a callout task to generate periodic beacons: m_dup(avp->beacon, M_DONTWAIT); mtx_lock(...); STAILQ_INSERT_TAIL(...); taskqueue_enqueue(...); mtx_unlock(...); callout_schedule(...); On the RX side, the interrupt handler will read out buffer then place it on a queue to be handled by wlan-glue code. For now wlan-glue code just frees the mbuf it instead of calling net80211 ieee80211_input() functions: m_copyback(...); /* Allocate new mbuf for next RX. */ MGETHDR(..., M_DONTWAIT, MT_DATA); bzero((mtod(sc->Rx_m, void *)), MHLEN); sc->Rx_m->m_len = 0; /* NB: m_gethdr does not set */ sc->Rx_m->m_data += 20; /* make headroom */ Then I use a lockmgr inside my kernel module that should make sure that we either are on TX or RX path. >>> >>> Uh, you can't use a lockmgr lock in interrupt handlers or in >>> if_transmit/if_start routines. You should most likely just be using a plain >>> mutex instead. Also, new code shouldn't use lockmgr in general. If you >>> need a sleepable lock, use sx instead. It has a more straightforward API. >> >> Ok, I will change the interrupt handler to do something like this: >> >> disaple_interrupt(); >> taskqueue_enqueue(...); /* on new rx task queue */ >> >> Then on the new rx proc: >> >> sx_slock(...); >> m_copyback(...); >> enable_interrupt(); >> /* Allocate new mbuf for next RX. */ >> MGETHDR(..., M_DONTWAIT, MT_DATA); >> bzero((mtod(sc->Rx_m, void *)), MHLEN); >> sc->Rx_m->m_len = 0; /* NB: m_gethdr does not set */ >> sc->Rx_m->m_data += 20; /* make headroom */ >> sx_sunlock(...); >> >> I lock TX/RX paths to make sure my code is threading safe. >> Also because while programming my deivce (SPI communicatioin) >> there will be a tsleep() waiting for the DMA interrupt and >> thus we could be prempted by e.g. a beacon_callout etc... >> > > I did implement your suggestions, using sx and modified interrupt handler > as specified above. But still same problem as before. > >>> The problem seems to be at [2], somehow after swapping buckets in uma_zfree m_dup returns a pointer to an mbuf that is still being used by us, [1] and [3] have same address. Then we call m_freem twice on same mbuf, [4] and [5]. And a loop occurs inside uma_free. I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? Problem seems to occur while swapping buckets. >>> >>> Hmm, the uma uses its own locking, so it should be safe, yes. However, you >>> are correct about [1] and [3]. The thing is, after [1] the mbuf shouldn't >>> be in any buckets at all (it only gets put back into the bucket during a >>> free). Are you sure the mbuf wasn't double free'd previously? > > I rechecked and it is almost certain that I dont double free the mbuf > before [1]. > And its not like it crashed in the beginning, it does run for a while > and then it crashes. So our code works for like a hundred beacons > sent/received > between two arm boards. Its feels like something is preempted, which explains > why the mbuf is still in the bucket (wrongly)? > are you running an INVARIANTS/DIAGNOSTICS/WITNESS/LOCK_DEBUG/... enabled kernel ? are you running on an SMP platform where there might be cache-coherency issue ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
Hi, "Mark Linimon" pisze: > We don't have a way to track emails that various users send to individual > maintainers. With a PR open, we have a way to do that. We also track > maintainer-timeouts, and these can eventually lead to a maintainer reset. > It is less a problem of having the tools than having the will to do everything publicly. From experience, committers loves to do all kind of things privately/secretly. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
Hi, On Tue, Jan 24, 2012 at 4:10 PM, Mark Linimon wrote: >> On 24 January 2012 18:36, Arnaud Lacombe wrote: >> It is less a problem of having the tools than having the will to do >> everything publicly. From experience, committers loves to do all kind >> of things privately/secretly. > > Perhaps it's human nature because of all the increasingly nit-picky, passive- > agressive, whiny, sarcastic, and generally asinine postings such as yours. > You are indeed right. However, I might just be also interested to review/comment code, discuss regressions, and architecture, for a change ;-) Unfortunately, such threads rarely ever happens. Most of the time, the only food provided is a really indigestible +5000/-3000 patch, where all the thinking, architectural design and code has been done behind closed door of a limited few committers, research lab or company. regards. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
Hi, On Wed, Jan 25, 2012 at 2:28 AM, Mark Linimon wrote: >> I might just be also interested to review/comment code, discuss >> regressions, and architecture, for a change ;-) >> Unfortunately, such threads rarely ever happen. Most of the time, the >> only food provided is a really indigestible +5000/-3000 patch, where >> all the thinking, architectural design and code has been done behind >> closed door of a limited few committers, research lab or company. > > That's odd. What the src committers usually tell me, when I have my > bugmeister-advocate hat on, is that they post patches and then no one > comments until after they check them in, at which time they complain. > This discourages them from going through this the next time. > exactly my point, huge patches are impossible to review. > You will also note that some of the large commits say "MFp4" or "MF: > ". That means that either our Perforce repository, or > SVN project/ directory, were used as staging areas. It's possible to > subscribe to these email messages. (Exactly how is left as an exercise > for the reader; the hour is getting late.) > that is indeed a good source for having a look at early-alpha-WIP stuff. > As for the research lab/company commits, I'm sure you'd complain equally > if the code that these groups develop in-house and then release when it's > in some kind of stable state, instead didn't get released at all. > I see company contributed code as ad-hoc solution to the company's problem, not general solution for the whole FreeBSD userbase. To make a comparison with Linux, it is just as if Google got all the Android code merged in mainline as-is, without re-working anything. It did not happen that way. Much of their code had (and still has) to be re-designed, abstracted, and adapted to fit the general-purpose mainline. > But, of course, I'm wasting my time trying to give you reasoned arguments > about why FreeBSD does one thing or another. AFAICT you're only interested > in spreading FUD about what we do, how we do it, and what we say about it > before, during, and afterwards. is this FUD: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/160992 ? is this FUD: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/156540 ? is this FUD: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/156799 ? is this FUD: http://lists.freebsd.org/pipermail/freebsd-current/2011-September/027400.html ? is this FUD: http://lists.freebsd.org/pipermail/freebsd-current/2011-December/030076.html ? answer to all the above: no, this is bugs, regressions, and mis-design you folks introduced, not me. Don't blame me to point it out. > You seem to be obsessed by picking over > semantics and finding shortcomings to be aggreived over. > Semantics and proper, independent, API are crucial. There is tons of ad-hoc code in FreeBSD which should be properly generalized. The most silly example I can think about is `time_after()', defined in . This has _nothing_ to do specifically with IEEE802.11, it's about time manipulation. Feel free to search the tree, there is tons of potentially unsafe, open-coded version of this macros. Call it nit-picking if you want, but when I write code, I want an API to use, I'm fed up to always have to re-invent the wheel. Btw, I do not even speak about some functions in the kernel re-implementing the exact same logic +10 times in a row, one after the other, within the same function body... For the story, I've been hacking tonight in Linux... a pure pleasure, real tough to get to, but really enjoyable. > Whatever patches or review you've contributed to date, to my mind, are > like the last tiny little bits of onion that are left over after one peels > off all the outer layers. There may be something to it, but the effort > to get down to that point is so painful that it's not worth it. > > tl;dr: your drama outweighs your contributions. > I already commented on this. I'm no longer interested in getting my stuff integrated in FreeBSD. I put it on github, eventually send patches on MLs, then you do what you want with it, I no longer particularly care. I know some patches are used around, that's enough. I did my time fighting committers to fix their not-so-bugfree code and won those battles, that's enough for me. - Arnaud ps: I have a particular appreciation for this PR, a feature praised by users, and no committer dares to care: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/161553 ... silly. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: FreeBSD has serious problems with focus, longevity, and lifecycle
Hi, On Wed, Jan 25, 2012 at 3:22 PM, Mark Linimon wrote: > On Wed, Jan 25, 2012 at 03:58:41AM -0500, Arnaud Lacombe wrote: >> > You seem to be obsessed by picking over >> > semantics and finding shortcomings to be aggreived over. >> > >> Semantics and proper, independent, API are crucial. > > I'm talking about the semantics of the non-technical part: the wording > of things that people post in email. viz: the time-wasting nonsense > about "oh, it seems to be released already, has anyone noticed?" from > a few weeks ago. > > It was 100% FUD, which apparently you knew perfectly well at the time, > and thus AFAICT only posted it to draw attention to yourself. > > That's the drama I'm talking about. > you are misquoting and misinterpreting me, I merely asked "Has 9.0 been released ?", followed by misleading clues I tried to gather in a thread where the original poster's uname shown "9-RELEASE". I see no FUD in this and none was intended. The rest of the thread indeed went ballistic. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Panic in 8.1 in softclock(): mutex Giant not owned at ../../../../sys/kern/kern_mutex.c:185
Hi,
Vijay Singh wrote;
> In the code below:
> (kgdb-amd64-6.8-48) l
> 402 if (c_lock != NULL) {
> 403 if (c_lock ==
> &Giant.lock_object) {
> 404 mtx_lock(&Giant);
> 405 mtx_assert(&Giant,
> MA_OWNED);
> 406 } else {
> 407
> class->lc_lock(c_lock, sharedlock);
> 408 }
>
Out of curiosity, where does your code comes from ? The same snippet
from the `release/8.1.0' branch does not refer to `Giant' at all, it
only reads:
if (c_lock != NULL) {
class->lc_lock(c_lock, sharedlock);
Thanks,
- Arnaud
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: > On 02/15/12 21:54, Jeff Roberson wrote: >> >> On Wed, 15 Feb 2012, Alexander Motin wrote: >>> >>> I've decided to stop those cache black magic practices and focus on >>> things that really exist in this world -- SMT and CPU load. I've >>> dropped most of cache related things from the patch and made the rest >>> of things more strict and predictable: >>> http://people.freebsd.org/~mav/sched.htt34.patch >> >> >> This looks great. I think there is value in considering the other >> approach further but I would like to do this part first. It would be >> nice to also add priority as a greater influence in the load balancing >> as well. > > > I haven't got good idea yet about balancing priorities, but I've rewritten > balancer itself. As soon as sched_lowest() / sched_highest() are more > intelligent now, they allowed to remove topology traversing from the > balancer itself. That should fix double-swapping problem, allow to keep some > affinity while moving threads and make balancing more fair. I did number of > tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 > threads everything is stationary as it should. With 9 threads I see regular > and random load move between all 8 CPUs. Measurements on 5 minutes run show > deviation of only about 5 seconds. It is the same deviation as I see caused > by only scheduling of 16 threads on 8 cores without any balancing needed at > all. So I believe this code works as it should. > > Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch > > I plan this to be a final patch of this series (more to come :)) and if > there will be no problems or objections, I am going to commit it (except > some debugging KTRs) in about ten days. So now it's a good time for reviews > and testing. :) > is there a place where all the patches are available ? I intend to run some tests on a 1x2x2 (atom D510), 1x4x1 (core-2 quad), and eventually a 2x8x2 platforms, against r231573. Results should hopefully be available by the end of the week-end/middle of next week[0]. - Arnaud [0]: the D510 will likely be testing a couple of Linux kernel over the week-end, and a FreeBSD run takes about 2.5 days to complete. > > -- > Alexander Motin > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: geom_aes
Hi, On Tue, Feb 21, 2012 at 8:00 AM, Ivan Voras wrote: > On 20/02/2012 01:14, Oliver Pinter wrote: >> Hi all! >> >> What is geom_aes (sys/geom/geom_aes.*) (wherein differs form geli, >> besides useses fewer geom layer and has a compiled in key), and who >> can I find something from this geom module/layer? >> >> How can I use this geom layer? Only added option geom_aes to kernel config? > > It is an old, example GEOM module, and should not be used at all, ever. > The same is true for geom_bde. > why ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Freebsd 9.0 can't detect ethernet card
Hi, On Mon, Mar 5, 2012 at 7:53 AM, Mark Blackman wrote: > > On 5 Mar 2012, at 12:30, Damien Fleuriot wrote: > >> Hello Elman, >> >> >> None of us was born a seer so you might want to tell the model of the >> network card. >> >> >> >> On 3/5/12 12:16 PM, Elman wrote: >>> Hallo hacker. >>> >>> I try install freebsd 9.0 in server Hp proliant ML370 G6, in process >>> install, freebsd can't detect ethernet card in automatic. Freebsd doesn't >>> support the ethernet card driver in HP ML370 G6? > > http://h30499.www3.hp.com/t5/ProLiant-Servers-Netservers/NC375i-on-Proliant-Server/td-p/1116100 > > Suggests its a Qlogic NetXen3 but I can't immediately tell if that's > supported. It looks like > it's *not*, but HP is a rather big vendor not to have support for, so I'm > guessing I've just > overlooked it > PCI vendor and device ID would be much more useful than brand name. Elman, could you send the output of `dmesg', as well as `pciconf -l' ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Freebsd 9.0 can't detect ethernet card
Hi, [Thanks folks to have removed the original sender from the CC: list... Adding him back.] On Mon, Mar 5, 2012 at 10:50 AM, Arnaud Lacombe wrote: > Hi, > > On Mon, Mar 5, 2012 at 7:53 AM, Mark Blackman wrote: >> >> On 5 Mar 2012, at 12:30, Damien Fleuriot wrote: >> >>> Hello Elman, >>> >>> >>> None of us was born a seer so you might want to tell the model of the >>> network card. >>> >>> >>> >>> On 3/5/12 12:16 PM, Elman wrote: >>>> Hallo hacker. >>>> >>>> I try install freebsd 9.0 in server Hp proliant ML370 G6, in process >>>> install, freebsd can't detect ethernet card in automatic. Freebsd doesn't >>>> support the ethernet card driver in HP ML370 G6? >> >> http://h30499.www3.hp.com/t5/ProLiant-Servers-Netservers/NC375i-on-Proliant-Server/td-p/1116100 >> >> Suggests its a Qlogic NetXen3 but I can't immediately tell if that's >> supported. It looks like >> it's *not*, but HP is a rather big vendor not to have support for, so I'm >> guessing I've just >> overlooked it >> > PCI vendor and device ID would be much more useful than brand name. > Elman, could you send the output of `dmesg', as well as `pciconf -l' ? > > Thanks, > - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin : > On 17.02.2012 18:53, Arnaud Lacombe wrote: >> >> On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin wrote: >>> >>> On 02/15/12 21:54, Jeff Roberson wrote: >>>> >>>> On Wed, 15 Feb 2012, Alexander Motin wrote: >>>>> >>>>> I've decided to stop those cache black magic practices and focus on >>>>> things that really exist in this world -- SMT and CPU load. I've >>>>> dropped most of cache related things from the patch and made the rest >>>>> of things more strict and predictable: >>>>> http://people.freebsd.org/~mav/sched.htt34.patch >>>> >>>> >>>> This looks great. I think there is value in considering the other >>>> approach further but I would like to do this part first. It would be >>>> nice to also add priority as a greater influence in the load balancing >>>> as well. >>> >>> >>> I haven't got good idea yet about balancing priorities, but I've >>> rewritten >>> balancer itself. As soon as sched_lowest() / sched_highest() are more >>> intelligent now, they allowed to remove topology traversing from the >>> balancer itself. That should fix double-swapping problem, allow to keep >>> some >>> affinity while moving threads and make balancing more fair. I did number >>> of >>> tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and >>> 16 >>> threads everything is stationary as it should. With 9 threads I see >>> regular >>> and random load move between all 8 CPUs. Measurements on 5 minutes run >>> show >>> deviation of only about 5 seconds. It is the same deviation as I see >>> caused >>> by only scheduling of 16 threads on 8 cores without any balancing needed >>> at >>> all. So I believe this code works as it should. >>> >>> Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch >>> >>> I plan this to be a final patch of this series (more to come :)) and if >>> there will be no problems or objections, I am going to commit it (except >>> some debugging KTRs) in about ten days. So now it's a good time for >>> reviews >>> and testing. :) >>> >> is there a place where all the patches are available ? > > > All my scheduler patches are cumulative, so all you need is only the last > mentioned here sched.htt40.patch. > You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. > But in some cases, especially for multi-socket systems, to let it show its > best, you may want to apply additional patch from avg@ to better detect CPU > topology: > https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd > test I conducted specifically for this patch did not showed much improvement... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, 2012/4/9 Alexander Motin : > [...] > > I have strong feeling that while this test may be interesting for profiling, > it's own results in first place depend not from how fast scheduler is, but > from the pipes capacity and other alike things. Can somebody hint me what > except pipe capacity and context switch to unblocked receiver prevents > sender from sending all data in batch and then receiver from receiving them > all in batch? If different OSes have different policies there, I think > results could be incomparable. > Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y > X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Tue, Apr 10, 2012 at 1:53 PM, Alexander Motin wrote: > On 04/10/12 20:18, Alexander Motin wrote: >> >> On 04/10/12 19:58, Arnaud Lacombe wrote: >>> >>> 2012/4/9 Alexander Motin: >>>> >>>> [...] >>>> >>>> I have strong feeling that while this test may be interesting for >>>> profiling, >>>> it's own results in first place depend not from how fast scheduler >>>> is, but >>>> from the pipes capacity and other alike things. Can somebody hint me >>>> what >>>> except pipe capacity and context switch to unblocked receiver prevents >>>> sender from sending all data in batch and then receiver from >>>> receiving them >>>> all in batch? If different OSes have different policies there, I think >>>> results could be incomparable. >>>> >>> Let me disagree on your conclusion. If OS A does a task in X seconds, >>> and OS B does the same task in Y seconds, if Y> X, then OS B is just >>> not performing good enough. Internal implementation's difference for >>> the task can not be waived as an excuse for result's comparability. >> >> >> Sure, numbers are always numbers, but the question is what are they >> showing? Understanding of the test results is even more important for >> purely synthetic tests like this. Especially when one test run gives 25 >> seconds, while another gives 50. This test is not completely clear to me >> and that is what I've told. > > Small illustration to my point. Simple scheduler tuning affects thread > preemption policy and changes this test results in three times: > > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 9.568 > > mav@test:/test/hackbench# sysctl kern.sched.interact=0 > kern.sched.interact: 30 -> 0 > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 5.163 > > mav@test:/test/hackbench# sysctl kern.sched.interact=100 > kern.sched.interact: 0 -> 100 > mav@test:/test/hackbench# ./hackbench 30 process 1000 > Running with 30*40 (== 1200) tasks. > Time: 3.190 > > I think it affects balance between pipe latency and bandwidth, while test > measures only the last. It is clear that conclusion from these numbers > depends on what do we want to have. > I don't really care on this point, I'm only testing default values, or more precisely, whatever developers though default values would be good. Btw, you are testing 3 differents configuration. Different results are expected. What worries me more is the rather the huge instability on the *same* configuration, say on a pipe/thread/70 groups/600 iterations run, where results range from 2.7s[0] to 7.4s, or a socket/thread/20 groups/1400 iterations run, where results range from 2.4s to 4.5s. - Arnaud [0]: numbers extracted from a recent run of 9.0-RELEASE on a Xeon E5-1650 platform. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Tue, Apr 10, 2012 at 4:05 PM, Mike Meyer wrote: > On Tue, 10 Apr 2012 12:58:00 -0400 > Arnaud Lacombe wrote: >> Let me disagree on your conclusion. If OS A does a task in X seconds, >> and OS B does the same task in Y seconds, if Y > X, then OS B is just >> not performing good enough. > > Others have pointed out one problem with this statement. Let me point > out another: > > It ignores the purpose of the system. If you change the task to doing > N concurrent versions of the task, and OS A time increases linearly > with the number of tasks (say it's time X*N) but OS B stair-steps at > the number of processors in the system (i.e. Y*floor(N/P)), then OS A > is just not performing good enough. > > A more concrete example: if OS B spends a couple of microseconds > optimizing disk access order and OS A doesn't, then a single process > writing to disk on OS A could well run faster than the same on OS > B. However, the maximum throughput on OS B as you add process will be > higher than it is on OS A. Which one you want will depend on what > you're using the system for. > You are discussing implementations in both case. If the implementation is not good enough, let's improve it, but do not discard the numbers on false claims. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Disabling an arbitrary device
Hi, I will be bringing up an old thread there, but it would seem the situation did not evolve in the past 9 years. I have a machine running 7.1 whose UHCI controller is generating some interrupt storm: # vmstat -i interrupt total rate irq4: sio0 1328 2 irq19: uhci1+ 63514509 96380 [...] generating useless load on one CPU: # top -SH last pid: 5223; load averages: 0.00, 0.00, 0.00up 0+00:17:21 13:10:35 117 processes: 14 running, 79 sleeping, 24 waiting CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free [...] 57 root -64- 0K 8K CPU0 0 11:59 86.57% irq19: uhci1+ I thought I could use an hint to forbid uhci(4) attachment, ala: hint.uhci.0.disabled="1" hint.uhci.1.disabled="1" in /boot/loader.conf. However, it would seem that what should be usable with any arbitrary devices, ie. be an integral part of device(9), has to be hardcoded in every driver, sad. - Arnaud ps: the original thread I found is from September 2004: http://lists.freebsd.org/pipermail/freebsd-questions/2004-September/058717.html ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Disabling an arbitrary device
Hi, On Fri, Apr 20, 2012 at 2:16 PM, Arnaud Lacombe wrote: > Hi, > > I will be bringing up an old thread there, but it would seem the > situation did not evolve in the past 9 years. I have a machine running > 7.1 whose UHCI controller is generating some interrupt storm: > > # vmstat -i > interrupt total rate > irq4: sio0 1328 2 > irq19: uhci1+ 63514509 96380 > [...] > > generating useless load on one CPU: > > # top -SH > last pid: 5223; load averages: 0.00, 0.00, 0.00 up 0+00:17:21 > 13:10:35 > 117 processes: 14 running, 79 sleeping, 24 waiting > CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle > Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free > [...] > 57 root -64 - 0K 8K CPU0 0 11:59 86.57% irq19: uhci1+ > > I thought I could use an hint to forbid uhci(4) attachment, ala: > > hint.uhci.0.disabled="1" > hint.uhci.1.disabled="1" > > in /boot/loader.conf. However, it would seem that what should be > usable with any arbitrary devices, ie. be an integral part of > device(9), has to be hardcoded in every driver, sad. > as for the usual "if you're not happy, please provide a patch": https://github.com/lacombar/freebsd/commit/30786d09b0cb441890cdc749ee5243238e81d2d8 regards, - Arnaud > - Arnaud > > ps: the original thread I found is from September 2004: > http://lists.freebsd.org/pipermail/freebsd-questions/2004-September/058717.html ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Disabling an arbitrary device
Hi, On Fri, Apr 20, 2012 at 8:50 PM, Attilio Rao wrote: > Il 20 aprile 2012 19:18, Arnaud Lacombe ha scritto: >> Hi, >> >> On Fri, Apr 20, 2012 at 2:16 PM, Arnaud Lacombe wrote: >>> Hi, >>> >>> I will be bringing up an old thread there, but it would seem the >>> situation did not evolve in the past 9 years. I have a machine running >>> 7.1 whose UHCI controller is generating some interrupt storm: >>> >>> # vmstat -i >>> interrupt total rate >>> irq4: sio0 1328 2 >>> irq19: uhci1+ 63514509 96380 >>> [...] >>> >>> generating useless load on one CPU: >>> >>> # top -SH >>> last pid: 5223; load averages: 0.00, 0.00, 0.00 up 0+00:17:21 >>> 13:10:35 >>> 117 processes: 14 running, 79 sleeping, 24 waiting >>> CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle >>> Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free >>> [...] >>> 57 root -64 - 0K 8K CPU0 0 11:59 86.57% irq19: >>> uhci1+ >>> >>> I thought I could use an hint to forbid uhci(4) attachment, ala: >>> >>> hint.uhci.0.disabled="1" >>> hint.uhci.1.disabled="1" >>> >>> in /boot/loader.conf. However, it would seem that what should be >>> usable with any arbitrary devices, ie. be an integral part of >>> device(9), has to be hardcoded in every driver, sad. >>> >> as for the usual "if you're not happy, please provide a patch": >> >> https://github.com/lacombar/freebsd/commit/30786d09b0cb441890cdc749ee5243238e81d2d8 > > I think a generalized kludge should really belong to > device_probe_child() rather than device_add_child_ordered(). > suit me. > When you have a tested patch against -CURRENT, can you please send to > freebsd-newbus@? > will give it a try, eventually. > BTW, IIRC 7.x doesn't have the new USB stack, which means that you can > have caught easilly a driver bug there, did you consider switching at > least to 8.x kernel? > That is unfortunately not a decision for me to make, though a switch to 9.x will be more likely. That said, it seems that it is not a USB specific issue, but an IRQ19's issue. I did not had a chance to investigate that further, but with uhci(4) disabled, atapci(4) is now IRQ-storming. This is a known issue relatively widely encountered, with various workarounds possible. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Sun, Dec 11, 2011 at 8:08 AM, Ulrich Spörlein wrote: > On Sun, 2011-12-04 at 18:42:38 -0500, Arnaud Lacombe wrote: >> On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein wrote: >> > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: >> >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe >> >> wrote: >> >> > FWIW, how comes that there is not yet any `stable/9' branch on the >> >> > github tree ? >> >> > >> >> Ulrich, ping ? >> > >> > Oops, sorry for the delay! Fixed now, thanks for bringing it to my >> > attention. I missed the push --all flag. :/ >> > >> FWIW, the github mirror[0] of the completed SVN tree has not seen any >> upgrade for a week now. Did some script broke ? >> >> Thanks, >> - Arnaud >> >> [0]: https://github.com/freebsd/freebsd > > Yeah, sorry about that. Monitoring is not that great, so you should > always ping me directly. > > The problem is that people dump files at random places in the Subversion > tree and that cannot easily be translated to branches. This time it was > /vendor/flex/FreeBSD-Xlist > It would seem the full SVN mirror on github is out-of-sync again, at least, the `user/imp/extern_cc' is missing. Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Sat, May 12, 2012 at 3:34 AM, Ulrich Spörlein wrote: > On Fri, 2012-05-11 at 16:49:42 -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Sun, Dec 11, 2011 at 8:08 AM, Ulrich Spörlein wrote: >> > On Sun, 2011-12-04 at 18:42:38 -0500, Arnaud Lacombe wrote: >> >> On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein >> >> wrote: >> >> > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: >> >> >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe >> >> >> wrote: >> >> >> > FWIW, how comes that there is not yet any `stable/9' branch on the >> >> >> > github tree ? >> >> >> > >> >> >> Ulrich, ping ? >> >> > >> >> > Oops, sorry for the delay! Fixed now, thanks for bringing it to my >> >> > attention. I missed the push --all flag. :/ >> >> > >> >> FWIW, the github mirror[0] of the completed SVN tree has not seen any >> >> upgrade for a week now. Did some script broke ? >> >> >> >> Thanks, >> >> - Arnaud >> >> >> >> [0]: https://github.com/freebsd/freebsd >> > >> > Yeah, sorry about that. Monitoring is not that great, so you should >> > always ping me directly. >> > >> > The problem is that people dump files at random places in the Subversion >> > tree and that cannot easily be translated to branches. This time it was >> > /vendor/flex/FreeBSD-Xlist >> > >> It would seem the full SVN mirror on github is out-of-sync again, at >> least, the `user/imp/extern_cc' is missing. > > user/ isn't pushed to github (and I consider also not pushing projects/ > into it), as their webfrontend often craps out with too many branches. > > You can get it from git.freebsd.org directly. > ok, there seem to be a valid repository at `git://git.freebsd.org/freebsd'. It might take some time to clone it from BSDCan, so I'll wait to be back to base to clone. It sounds I'll need to re-export/re-import all my branches.. again Thanks for the pointer. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Sat, May 12, 2012 at 10:27 AM, Arnaud Lacombe wrote: > Hi, > > On Sat, May 12, 2012 at 3:34 AM, Ulrich Spörlein wrote: >> On Fri, 2012-05-11 at 16:49:42 -0400, Arnaud Lacombe wrote: >>> Hi, >>> >>> On Sun, Dec 11, 2011 at 8:08 AM, Ulrich Spörlein wrote: >>> > On Sun, 2011-12-04 at 18:42:38 -0500, Arnaud Lacombe wrote: >>> >> On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein >>> >> wrote: >>> >> > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: >>> >> >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe >>> >> >> wrote: >>> >> >> > FWIW, how comes that there is not yet any `stable/9' branch on the >>> >> >> > github tree ? >>> >> >> > >>> >> >> Ulrich, ping ? >>> >> > >>> >> > Oops, sorry for the delay! Fixed now, thanks for bringing it to my >>> >> > attention. I missed the push --all flag. :/ >>> >> > >>> >> FWIW, the github mirror[0] of the completed SVN tree has not seen any >>> >> upgrade for a week now. Did some script broke ? >>> >> >>> >> Thanks, >>> >> - Arnaud >>> >> >>> >> [0]: https://github.com/freebsd/freebsd >>> > >>> > Yeah, sorry about that. Monitoring is not that great, so you should >>> > always ping me directly. >>> > >>> > The problem is that people dump files at random places in the Subversion >>> > tree and that cannot easily be translated to branches. This time it was >>> > /vendor/flex/FreeBSD-Xlist >>> > >>> It would seem the full SVN mirror on github is out-of-sync again, at >>> least, the `user/imp/extern_cc' is missing. >> >> user/ isn't pushed to github (and I consider also not pushing projects/ >> into it), as their webfrontend often craps out with too many branches. >> >> You can get it from git.freebsd.org directly. >> > ok, there seem to be a valid repository at > `git://git.freebsd.org/freebsd'. It might take some time to clone it > from BSDCan, so I'll wait to be back to base to clone. It sounds I'll > need to re-export/re-import all my branches.. again > ok, that repository looks complete... the drawback of using it is that I'll lost the possibility to share anything on github or gitorious... Could it be thought to have all git mirror around (gitorious, github, your.org, ...), to actually be simple clone of this specific repository, to be able to facilitate code sharing among them ?? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Fwd: my git development snapshot(s)
Hi, On Sun, May 13, 2012 at 12:59 AM, Arnaud Lacombe wrote: > Hi, > > On Sat, May 12, 2012 at 10:27 AM, Arnaud Lacombe wrote: >> Hi, >> >> On Sat, May 12, 2012 at 3:34 AM, Ulrich Spörlein wrote: >>> On Fri, 2012-05-11 at 16:49:42 -0400, Arnaud Lacombe wrote: >>>> Hi, >>>> >>>> On Sun, Dec 11, 2011 at 8:08 AM, Ulrich Spörlein >>>> wrote: >>>> > On Sun, 2011-12-04 at 18:42:38 -0500, Arnaud Lacombe wrote: >>>> >> On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein >>>> >> wrote: >>>> >> > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: >>>> >> >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe >>>> >> >> wrote: >>>> >> >> > FWIW, how comes that there is not yet any `stable/9' branch on the >>>> >> >> > github tree ? >>>> >> >> > >>>> >> >> Ulrich, ping ? >>>> >> > >>>> >> > Oops, sorry for the delay! Fixed now, thanks for bringing it to my >>>> >> > attention. I missed the push --all flag. :/ >>>> >> > >>>> >> FWIW, the github mirror[0] of the completed SVN tree has not seen any >>>> >> upgrade for a week now. Did some script broke ? >>>> >> >>>> >> Thanks, >>>> >> - Arnaud >>>> >> >>>> >> [0]: https://github.com/freebsd/freebsd >>>> > >>>> > Yeah, sorry about that. Monitoring is not that great, so you should >>>> > always ping me directly. >>>> > >>>> > The problem is that people dump files at random places in the Subversion >>>> > tree and that cannot easily be translated to branches. This time it was >>>> > /vendor/flex/FreeBSD-Xlist >>>> > >>>> It would seem the full SVN mirror on github is out-of-sync again, at >>>> least, the `user/imp/extern_cc' is missing. >>> >>> user/ isn't pushed to github (and I consider also not pushing projects/ >>> into it), as their webfrontend often craps out with too many branches. >>> >>> You can get it from git.freebsd.org directly. >>> >> ok, there seem to be a valid repository at >> `git://git.freebsd.org/freebsd'. It might take some time to clone it >> from BSDCan, so I'll wait to be back to base to clone. It sounds I'll >> need to re-export/re-import all my branches.. again >> > ok, that repository looks complete... the drawback of using it is that > I'll lost the possibility to share anything on github or gitorious... > Could it be thought to have all git mirror around (gitorious, github, > your.org, ...), to actually be simple clone of this specific > repository, to be able to facilitate code sharing among them ?? > answer to myself, the git://git.freebsd.org/freebsd and the repository on github seem to actually be the same tree. github just has less branches exported in it. That's a good news ! - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
sysctl filesystem ?
Hi folks, I find myself in a situation where I need to directly explore the sysctl(8) tree from my program. The tricky part is this: from `src/sbin/sysctl.c': /* * These functions uses a presently undocumented interface to the kernel * to walk the tree and get the type so it can print the value. * This interface is under work and consideration, and should probably * be killed with a big axe by the first person who can find the time. * (be aware though, that the proper interface isn't as obvious as it * may seem, there are various conflicting requirements. */ AFAIT, the whole interface used by sysctl(8) to explore the sysctl tree (ie. list, name, get description) is undocumented. This comment has been there for about, well... 17 years. No matter to say that it is highly unlikely anyone is ever gonna design that perfect interface. Right now, I am left with no choice but to figure out how that stuff work, which I foresee will be a real PITA. No choice ? Well, not so much. About 12 years ago a filesystem interface was written for sysctl, namely scfs(4). It was authored by Kelly Yancey and (?) Boris Popov. Unfortunately, time passed and no code is any longer publicly available. URLs are either no longer valid or password-protected. This interface would just be perfect for my use-case. No need to spend time decoding a prehistoric interface, no need to craft custom accessors. I would just have to use standard POSIX file interface... if the code was available. I was hoping some of the people on this list would either, have the scfs(4) code on their drives/tapes, or have a way to contact the original author(s) and thus have a chance to get that code ? Thanks in advance, - Arnaud ps: I know the code will certainly have to be fixed, but that is still the best option I've got so far. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: sysctl filesystem ?
Hi, On Mon, Jun 25, 2012 at 8:30 PM, Adam Vande More wrote: > On Mon, Jun 25, 2012 at 7:03 PM, Arnaud Lacombe wrote: >> >> Hi folks, >> >> I find myself in a situation where I need to directly explore the >> sysctl(8) tree from my program. The tricky part is this: > > There is this: > > http://svnweb.freebsd.org/base/releng/4.7/sys/miscfs/kernfs/ > yes, `kernfs' is mentioned in some of the thread about a sysctl filesystem as a potential code base, and has been used in such purpose. However, if I can avoid to re-design that wheel too, by getting access to scfs(4) code, I will. - Arnaud > -- > Adam Vande More ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: sysctl filesystem ?
On Mon, Jun 25, 2012 at 8:13 PM, Garrett Cooper wrote:
> On Mon, Jun 25, 2012 at 5:03 PM, Arnaud Lacombe wrote:
>> Hi folks,
>>
>> I find myself in a situation where I need to directly explore the
>> sysctl(8) tree from my program. The tricky part is this:
>>
>> from `src/sbin/sysctl.c':
>> /*
>> * These functions uses a presently undocumented interface to the kernel
>> * to walk the tree and get the type so it can print the value.
>> * This interface is under work and consideration, and should probably
>> * be killed with a big axe by the first person who can find the time.
>> * (be aware though, that the proper interface isn't as obvious as it
>> * may seem, there are various conflicting requirements.
>> */
>>
>> AFAIT, the whole interface used by sysctl(8) to explore the sysctl
>> tree (ie. list, name, get description) is undocumented. This comment
>> has been there for about, well... 17 years. No matter to say that it
>> is highly unlikely anyone is ever gonna design that perfect interface.
>> Right now, I am left with no choice but to figure out how that stuff
>> work, which I foresee will be a real PITA. No choice ? Well, not so
>> much. About 12 years ago a filesystem interface was written for
>> sysctl, namely scfs(4). It was authored by Kelly Yancey and (?) Boris
>> Popov. Unfortunately, time passed and no code is any longer publicly
>> available. URLs are either no longer valid or password-protected.
>>
>> This interface would just be perfect for my use-case. No need to spend
>> time decoding a prehistoric interface, no need to craft custom
>> accessors. I would just have to use standard POSIX file interface...
>> if the code was available.
>>
>> I was hoping some of the people on this list would either, have the
>> scfs(4) code on their drives/tapes, or have a way to contact the
>> original author(s) and thus have a chance to get that code ?
>>
>> Thanks in advance,
>> - Arnaud
>>
>> ps: I know the code will certainly have to be fixed, but that is still
>> the best option I've got so far.
>
> Alternatively, the interface could just be documented (from
> sys/kern/kern_sysctl.c):
>
> /*
> * "Staff-functions"
> *
> * These functions implement a presently undocumented interface
> * used by the sysctl program to walk the tree, and get the type
> * so it can print the value.
> * This interface is under work and consideration, and should probably
> * be killed with a big axe by the first person who can find the time.
> * (be aware though, that the proper interface isn't as obvious as it
> * may seem, there are various conflicting requirements.
> *
> * {0,0} printf the entire MIB-tree.
> * {0,1,...} return the name of the "..." OID.
> * {0,2,...} return the next OID.
> * {0,3} return the OID of the name in "new"
> * {0,4,...} return the kind & format info for the "..." OID.
> * {0,5,...} return the description the "..." OID.
> */
>
> I know 2 closed-source versions that have wholesale stolen/"borrowed"
> the code from sysctl(3), and I adapted the sysctl(8) code for my own
> uses for the cython derivative I made here [1].
>
I guess I will have no choice but to write a third...
- Arnaud
> Thanks,
> -Garrett
>
> 1. http://sourceforge.net/projects/sysctl/
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: sysctl filesystem ?
Hi, On Mon, Jun 25, 2012 at 10:51 PM, Boris Popov wrote: > On 26.06.2012 6:56, Arnaud Lacombe wrote: >> purpose. However, if I can avoid to re-design that wheel too, by >> getting access to scfs(4) code, I will. > > It is interesting, that the old drive with this code are still alive. > Most likely, FS related part will need serious attention because of > numerous changes in the VFS subsystem. Here is the link: > > http://www.vertex.kz/scfs.tgz > Outstanding ! I was pointed another implementation by Jakub Dawidek made in 2002, which seems more advanced. In any case, lots of KPI changed... I guess I found a new toy for this week :-) Thanks a lot, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Interfacing devices with multiple parents within newbus
Hi folks,
The problem has been raised in the last BSDCan during a talk, but no
clear answer has been given. Some (pseudo-)devices might require
resources from multiple other (pseudo-)devices.
For example, a device is sitting on an SMBus, but need to access a
software controlled LED, sitting on a GPIO bus, itself sitting on an
LPC bus... Or a variant where everything is controlled on the same
chip, but different GPIO banks. For that specific example, all the
GPIO pin could be implement on the same GPIObus, however, gpiobus(4)
is limited to 32 pins and the chip provides 5 banks of 8 pins, ie. a
total of 40 pins[0]. In the same idea, a device sitting on GPIOs
controlled by two independant chips, say one being an ICH10R chipset
on a PCI device function, and the other being a
SuperIO on an LPC bus.
This situation make me really dubious that FreeBSD will be able to
support configuration such as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = <&gpio2 13 0>; /* GPIO3_13 */
wp-gpios = <&gpio3 11 0>; /* GPIO4_11 */
status = "okay";
};
or:
ecspi@5001 { /* ECSPI1 */
fsl,spi-num-chipselects = <2>;
cs-gpios = <&gpio1 30 0>, /* GPIO2_30 */
<&gpio2 19 0>; /* GPIO3_19 */
status = "okay";
[...]
This example is taken from Linux' `arch/arm/boot/dts/imx53-smd.dts'.
Here, SDHC or SPI controller are using different GPIO devices. Note
that these GPIO pins does not seem to be multi-function pins as
another .dts defines ESDHC1 as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = <&gpio2 13 0>; /* GPIO3_13 */
wp-gpios = <&gpio2 14 0>; /* GPIO3_14 */
status = "okay";
};
AFAIK, newbus is unable to model any of the above situation without
being bypassed one way or another.
any hints ?
Thanks,
- Arnaud
[0]: as a matter of comparison, old PXA27x have up to 121 GPIOs
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Interfacing devices with multiple parents within newbus
Hi,
On Fri, Jul 6, 2012 at 1:04 AM, Warner Losh wrote:
>
> On Jul 5, 2012, at 5:14 PM, Arnaud Lacombe wrote:
>
>> Hi folks,
>>
>> The problem has been raised in the last BSDCan during a talk, but no
>> clear answer has been given. Some (pseudo-)devices might require
>> resources from multiple other (pseudo-)devices.
>>
>> For example, a device is sitting on an SMBus, but need to access a
>> software controlled LED, sitting on a GPIO bus, itself sitting on an
>> LPC bus... Or a variant where everything is controlled on the same
>> chip, but different GPIO banks. For that specific example, all the
>> GPIO pin could be implement on the same GPIObus, however, gpiobus(4)
>> is limited to 32 pins and the chip provides 5 banks of 8 pins, ie. a
>> total of 40 pins[0]. In the same idea, a device sitting on GPIOs
>> controlled by two independant chips, say one being an ICH10R chipset
>> on a PCI device function, and the other being a
>> SuperIO on an LPC bus.
>>
>> This situation make me really dubious that FreeBSD will be able to
>> support configuration such as:
>>
>> esdhc@50004000 { /* ESDHC1 */
>>cd-gpios = <&gpio2 13 0>; /* GPIO3_13 */
>>wp-gpios = <&gpio3 11 0>; /* GPIO4_11 */
>>status = "okay";
>> };
>>
>> or:
>>
>> ecspi@5001 { /* ECSPI1 */
>>fsl,spi-num-chipselects = <2>;
>>cs-gpios = <&gpio1 30 0>, /* GPIO2_30 */
>> <&gpio2 19 0>; /* GPIO3_19 */
>>status = "okay";
>> [...]
>>
>> This example is taken from Linux' `arch/arm/boot/dts/imx53-smd.dts'.
>> Here, SDHC or SPI controller are using different GPIO devices. Note
>> that these GPIO pins does not seem to be multi-function pins as
>> another .dts defines ESDHC1 as:
>>
>> esdhc@50004000 { /* ESDHC1 */
>>cd-gpios = <&gpio2 13 0>; /* GPIO3_13 */
>>wp-gpios = <&gpio2 14 0>; /* GPIO3_14 */
>>status = "okay";
>> };
>>
>> AFAIK, newbus is unable to model any of the above situation without
>> being bypassed one way or another.
>>
>> any hints ?
>
> That's not correct. You don't need a parent-child relationship in newbus to
> interact with another node in the tree. You can look up the other node by
> name, and then call functions based on finding that other device.
>
I assume you are talking about devclass_get_device()/device_find_child().
That's neither correct nor robust in a couple of way:
1) you have no guarantee a device unit will always give you the same resource.
2) there is no reference counting on the returned device.
3) there is no track record of the reference being given.
About (1), lower unit devices can fails to attach[0], thus newly
attached bus will now have a negative offset.
About (2) and (3), referenced device (think KLD) might go away and the
child will not be told. In this situation, I want the child to be
detached prior to its parent.
As such, looking up other node by name would fit in what I call
"bypassing newbus purpose". I might just as well export a damn
function pointer and make my life easier.
What would be need is an interface where the child need to have a way
to say "resources I need are not complete, tell me when a new device
in devclass Z is attached that I can check if it fills my need", ala:
- dev0 is probed
- dev0 tells it is present, but need unavailable resource in devclass
X and devclass Y
- dev0 is marked as pending
- dev1 is attached in devclass X
- dev0 is asked if dev1 would suit it
- dev0 answers negatively
- dev2 is attached in devclass X
- dev0 is asked if dev2 would suit it
- dev0 answers positively
- dev0 continues its probe, but still need unavailable resource in devclass Y
- dev3 is attached in devclass Y
- dev0 is asked if dev3 would suit it
- dev0 answers positively
- dev0 finishes its probe
- dev0 is attached
[...]
- dev3 is to be detached
- dev0 is detached
- dev3 is detached
- Arnaud
[0]: for many reason; hardware failure, configuration change (think
GPIO on multi-function pin)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Interfacing devices with multiple parents within newbus
Hi, On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe wrote: > That's neither correct nor robust in a couple of way: > 1) you have no guarantee a device unit will always give you the same > resource. this raises the following question: how can a device, today, figure out which parent in a given devclass would give it access to resources it needs. Say, you have gpiobus0 provided by a superio and gpiobus1 provided by the chipset and a LED on the chipset's GPIO. Now, say gpiobus0 attachment is conditional to some BIOS setting. How can you tell gpioled(4) to attach on the chipset provided GPIO without hardcoding unit number either way ? AFAIK, you can not. Even hints provided layout description is defeated. Each device in a given devclass need to have a set of unique attribute to allow a child to distinguish it from other potential parent in the same devclass... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Interfacing devices with multiple parents within newbus
Hi, On Fri, Jul 6, 2012 at 3:09 PM, Ian Lepore wrote: > On Fri, 2012-07-06 at 14:46 -0400, Arnaud Lacombe wrote: >> Hi, >> >> On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe wrote: >> > That's neither correct nor robust in a couple of way: >> > 1) you have no guarantee a device unit will always give you the same >> > resource. >> this raises the following question: how can a device, today, figure >> out which parent in a given devclass would give it access to resources >> it needs. >> >> Say, you have gpiobus0 provided by a superio and gpiobus1 provided by >> the chipset and a LED on the chipset's GPIO. Now, say gpiobus0 >> attachment is conditional to some BIOS setting. How can you tell >> gpioled(4) to attach on the chipset provided GPIO without hardcoding >> unit number either way ? >> >> AFAIK, you can not. >> >> Even hints provided layout description is defeated. Each device in a >> given devclass need to have a set of unique attribute to allow a child >> to distinguish it from other potential parent in the same devclass... >> >> - Arnaud > > Talking about a child being unable to choose the correct parent seems to > indicate that this whole problem is turned upside-down somehow; children > don't choose their parents. > actually, I think I was wrong, I thought device were attached to a devclass, but they are truly attached to a given device. My mistake. > Just blue-sky dreaming here on the fly... what we really have is a > resource-management problem. A device comes along that needs a GPIO > resource, how does it find and use that resource? > > Well, we have a resource manager, could that help somehow? Could a > driver that provides access to GPIO somehow register its availability so > that another driver can find and access it? The "resource" may be a > callable interface, it doesn't really matter, I'm just wondering if the > current rman stuff could be leveraged to help make the connection > between unrelated devices. I think that implies that there would have > to be something near the root of the hiearchy willing to be the > owner/manager of dynamic resources. > AFAIR, rman is mostly there to manage memory vs. i/o mapped resources. The more I think about it, the more FTD is the answer. The open question now being "how to map a flexible device structure (FTD) to a less flexible structure (Newbus)" :/ - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Interfacing devices with multiple parents within newbus
Hi,
On Sat, Jul 7, 2012 at 2:47 PM, Ian Lepore
wrote:
> On Fri, 2012-07-06 at 16:45 -0400, Arnaud Lacombe wrote:
>> Hi,
>>
>> On Fri, Jul 6, 2012 at 3:09 PM, Ian Lepore
>> wrote:
>> > On Fri, 2012-07-06 at 14:46 -0400, Arnaud Lacombe wrote:
>> >> Hi,
>> >>
>> >> On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe
>> >> wrote:
>> >> > That's neither correct nor robust in a couple of way:
>> >> > 1) you have no guarantee a device unit will always give you the same
>> >> > resource.
>> >> this raises the following question: how can a device, today, figure
>> >> out which parent in a given devclass would give it access to resources
>> >> it needs.
>> >>
>> >> Say, you have gpiobus0 provided by a superio and gpiobus1 provided by
>> >> the chipset and a LED on the chipset's GPIO. Now, say gpiobus0
>> >> attachment is conditional to some BIOS setting. How can you tell
>> >> gpioled(4) to attach on the chipset provided GPIO without hardcoding
>> >> unit number either way ?
>> >>
>> >> AFAIK, you can not.
>> >>
>> >> Even hints provided layout description is defeated. Each device in a
>> >> given devclass need to have a set of unique attribute to allow a child
>> >> to distinguish it from other potential parent in the same devclass...
>> >>
>> >> - Arnaud
>> >
>> > Talking about a child being unable to choose the correct parent seems to
>> > indicate that this whole problem is turned upside-down somehow; children
>> > don't choose their parents.
>> >
>> actually, I think I was wrong, I thought device were attached to a
>> devclass, but they are truly attached to a given device. My mistake.
>>
>> > Just blue-sky dreaming here on the fly... what we really have is a
>> > resource-management problem. A device comes along that needs a GPIO
>> > resource, how does it find and use that resource?
>> >
>> > Well, we have a resource manager, could that help somehow? Could a
>> > driver that provides access to GPIO somehow register its availability so
>> > that another driver can find and access it? The "resource" may be a
>> > callable interface, it doesn't really matter, I'm just wondering if the
>> > current rman stuff could be leveraged to help make the connection
>> > between unrelated devices. I think that implies that there would have
>> > to be something near the root of the hiearchy willing to be the
>> > owner/manager of dynamic resources.
>> >
>> AFAIR, rman is mostly there to manage memory vs. i/o mapped resources.
>> The more I think about it, the more FTD is the answer. The open
>> question now being "how to map a flexible device structure (FTD) to a
>> less flexible structure (Newbus)" :/
>>
>> - Arnaud
>
> Memory- and IO-mapped regions and IRQs are the only current uses of rman
> (that I know of), but it was designed to be fairly agnostic about the
> resources it manages. It just works with ranges of values (that it
> really doesn't know how to interpret at all), leaving lots of room to
> define new types of things it can manage.
>
> The downside is that it's designed to be used hierarchically in the
> context of newbus, specifically to help parents manage the resources
> that they are able to provide to their children. Trying to use it in a
> way that allows devices which are hierarchically unrelated to allocate
> resources from each other may amount to a square-peg/round-hole
> situation. But the alternative is writing a new facility to allow
> registration and allocation of resources using some sort symbolic method
> of representing the resources such that the new manager doesn't have to
> know much about what it's managing. I think it would be better to find
> a way to reuse what we've already got if that's possible.
>
> I think we have two semi-related aspects to this problem...
>
> How do we symbolically represent the resources that drivers can provide
> to each other? (FDT may be the answer; I don't know much about it.)
>
> How do devices use that symbolic representation to locate the provider
> of the resource, and how is the sharing of those resources managed?
>
I'd say FDT answer both question, take the DTS for "Freescale i.MX53
Smart Mobile Reference Design Board"[0],
We first have SoC generic definition in `imx53.dtsi':
soc {
compatible = "simple-bus";
newbus' ivar's limitation..
Hi folks, Ok, yet another Newbus' limitation. Assuming a device exports more than one interface, and one of its child has need to use more than one interface, each interfaces cannot register, concurrently, its own ivar. While I try to always have a single child per interface/resource, I need to keep some compatibility with the old way of doing thing (POLA wrt. drivers I cannot/will not convert and userland). So, it would have been nice if ivar had been per-interface, not global and unique to one device. Unless I am mistaken, ivar are the only way for a parent can transmit information to a child. I can not simply implement a new METHOD to get that ivar as the device implements multiple time the same function (actually, up to 4 time for one, 3 for the other, with possible crossovers...), each one physically distinct. Each child is being tied to a pair. Thus, I need to pass each child discriminator(s) for each interfaces right after having been *created*, which cannot be done later on. Of course, it is out-of-question to have crossover in the interfaces definitions. The best way I could achieve this currently is to pass the child's device to its parent, and do a lookup based on that pointer to get information I need, but erk my 1c, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi,
On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh wrote:
>
> On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
>> Ok, yet another Newbus' limitation. Assuming a device exports more
>> than one interface, and one of its child has need to use more than one
>> interface, each interfaces cannot register, concurrently, its own
>> ivar. While I try to always have a single child per
>> interface/resource, I need to keep some compatibility with the old way
>> of doing thing (POLA wrt. drivers I cannot/will not convert and
>> userland). So, it would have been nice if ivar had been per-interface,
>> not global and unique to one device.
>
> There's one pointer for the ivars. The bus code gets to determine what the
> ivar looks like, because the interface is totally private to the bus. So
> long as it returns the right thing for any key that's presented, it doesn't
> matter quite how things are done.
>
> So I'm not sure I understand what you are saying here.
>
dev0 implements two interfaces: A and B. dev1, child of dev0, needs to
use both interfaces. There is no generic way for dev0 to export
independent ivars for both interface. For now, I restricted the
function of the second interface not to need ivar, but that's kind of
hackish.
- Arnaud
> The problem, more basically, is that the ivar keys are not unique.
> Currently, there's no bits used in the key to define the values to be
> non-overlapping. For example:
> enum pci_device_ivars {
> PCI_IVAR_SUBVENDOR,
> PCI_IVAR_SUBDEVICE,
> PCI_IVAR_VENDOR,
>
> };
>
> We could easily reserve the upper 16-bits of this field to be that key. This
> value could then be used to differentiate them. But this wouldn't scale too
> well. Given that there's only about a dozen or two in the tree, that's right
> at the moment, it wouldn't be hard to do something like:
>
> enum ivar_namespace {
> IVAR_PCI = 1,
> IVAR_PCCARD,
> IVAR_USB,
> etc
> };
> #define IVAR_SHIFT 16
>
> and the above could be changed to:
>
> enum pci_device_ivars {
> PCI_IVAR_SUBVENDOR = IVAR_PCI << IVAR_SHIFT,
> PCI_IVAR_SUBDEVICE,
> PCI_IVAR_VENDOR,
>
> };
>
> and then we'd have an unambiguous key, and the bus could easily implement
> multiple interfaces.
>
> but then again, most of the existing interfaces in the kernel are mutually
> exclusive, so you could implement this just for your new interfaces.
>
>> Unless I am mistaken, ivar are the only way for a parent can transmit
>> information to a child. I can not simply implement a new METHOD to get
>> that ivar as the device implements multiple time the same function
>> (actually, up to 4 time for one, 3 for the other, with possible
>> crossovers...), each one physically distinct. Each child is being tied
>> to a pair. Thus, I need to pass each child discriminator(s) for each
>> interfaces right after having been *created*, which cannot be done
>> later on. Of course, it is out-of-question to have crossover in the
>> interfaces definitions.
>
> ivars are but one way to communicate this. However, they are the generic way
> to convert a key to a value and store a key on a value. I don't really
> understand what you are trying to say here, perhaps an example would help
> illustrate what you are trying to do, since I don't quite understand the
> problem here.
>
>> The best way I could achieve this currently is to pass the child's
>> device to its parent, and do a lookup based on that pointer to get
>> information I need, but erk
>
> That doesn't make any sense. The child's parent already sets that child's
> ivar when the child is created. The child's parent already gets a pointer to
> the child when asked to do the key to value translation. Again, perhaps an
> example would help here.
>
> Warner
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh wrote:
>
> On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote:
>
>> Hi,
>>
>> On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh wrote:
>>>
>>> On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
>>>> Ok, yet another Newbus' limitation. Assuming a device exports more
>>>> than one interface, and one of its child has need to use more than one
>>>> interface, each interfaces cannot register, concurrently, its own
>>>> ivar. While I try to always have a single child per
>>>> interface/resource, I need to keep some compatibility with the old way
>>>> of doing thing (POLA wrt. drivers I cannot/will not convert and
>>>> userland). So, it would have been nice if ivar had been per-interface,
>>>> not global and unique to one device.
>>>
>>> There's one pointer for the ivars. The bus code gets to determine what the
>>> ivar looks like, because the interface is totally private to the bus. So
>>> long as it returns the right thing for any key that's presented, it doesn't
>>> matter quite how things are done.
>>>
>>> So I'm not sure I understand what you are saying here.
>>>
>> dev0 implements two interfaces: A and B. dev1, child of dev0, needs to
>> use both interfaces. There is no generic way for dev0 to export
>> independent ivars for both interface. For now, I restricted the
>> function of the second interface not to need ivar, but that's kind of
>> hackish.
>
> Only if the IVARs for interface A and interface B have overlapping values.
> If the Ivar keys don't overlap, then there's no problems at all. Certainly
> less hackish than not using them at all. Since dev0 knows the layout of the
> ivar that it set on its child, this presents no problems at all. It would
> return the values from A from the right part of the ivar, and those from B in
> the right part. Apart from the coordination of Ivar numbers, as I outlined
> in my last post, there's no issue here.
>
I think we should not be talking about the same API here. I have no
idea what you mean by "the key to value translation", nor "Ivar
numbers". What I refer to is that device_set_ivars() /
device_get_ivars() acts on a single instance variables from `struct
device': `ivars'. In that case, I do not really see how to set that
specific field to two distinct values for each interfaces.
- Arnaud
> Warner
>
>> - Arnaud
>>
>>> The problem, more basically, is that the ivar keys are not unique.
>>> Currently, there's no bits used in the key to define the values to be
>>> non-overlapping. For example:
>>> enum pci_device_ivars {
>>>PCI_IVAR_SUBVENDOR,
>>>PCI_IVAR_SUBDEVICE,
>>>PCI_IVAR_VENDOR,
>>>
>>> };
>>>
>>> We could easily reserve the upper 16-bits of this field to be that key.
>>> This value could then be used to differentiate them. But this wouldn't
>>> scale too well. Given that there's only about a dozen or two in the tree,
>>> that's right at the moment, it wouldn't be hard to do something like:
>>>
>>> enum ivar_namespace {
>>>IVAR_PCI = 1,
>>>IVAR_PCCARD,
>>>IVAR_USB,
>>> etc
>>> };
>>> #define IVAR_SHIFT 16
>>>
>>> and the above could be changed to:
>>>
>>> enum pci_device_ivars {
>>>PCI_IVAR_SUBVENDOR = IVAR_PCI << IVAR_SHIFT,
>>>PCI_IVAR_SUBDEVICE,
>>>PCI_IVAR_VENDOR,
>>>
>>> };
>>>
>>> and then we'd have an unambiguous key, and the bus could easily implement
>>> multiple interfaces.
>>>
>>> but then again, most of the existing interfaces in the kernel are mutually
>>> exclusive, so you could implement this just for your new interfaces.
>>>
>>>> Unless I am mistaken, ivar are the only way for a parent can transmit
>>>> information to a child. I can not simply implement a new METHOD to get
>>>> that ivar as the device implements multiple time the same function
>>>> (actually, up to 4 time for one, 3 for the other, with possible
>>>> crossovers...), each one physically distinct. Each child is being tied
>>>> to a pair. Thus, I need to pass each child discriminator(s) for each
>>>> interfaces right after having been *created*, which cannot be done
>>>> later on. Of course, it is out-of-question to have crossove
Re: newbus' ivar's limitation..
Hi,
On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh wrote:
>
> On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
>> Ok, yet another Newbus' limitation. Assuming a device exports more
>> than one interface, and one of its child has need to use more than one
>> interface, each interfaces cannot register, concurrently, its own
>> ivar. While I try to always have a single child per
>> interface/resource, I need to keep some compatibility with the old way
>> of doing thing (POLA wrt. drivers I cannot/will not convert and
>> userland). So, it would have been nice if ivar had been per-interface,
>> not global and unique to one device.
>
> There's one pointer for the ivars. The bus code gets to determine what the
> ivar looks like, because the interface is totally private to the bus. So
> long as it returns the right thing for any key that's presented, it doesn't
> matter quite how things are done.
>
> So I'm not sure I understand what you are saying here.
>
> The problem, more basically, is that the ivar keys are not unique.
> Currently, there's no bits used in the key to define the values to be
> non-overlapping. For example:
> enum pci_device_ivars {
> PCI_IVAR_SUBVENDOR,
> PCI_IVAR_SUBDEVICE,
> PCI_IVAR_VENDOR,
>
> };
>
> We could easily reserve the upper 16-bits of this field to be that key. This
> value could then be used to differentiate them. But this wouldn't scale too
> well. Given that there's only about a dozen or two in the tree, that's right
> at the moment, it wouldn't be hard to do something like:
>
> enum ivar_namespace {
> IVAR_PCI = 1,
> IVAR_PCCARD,
> IVAR_USB,
> etc
> };
> #define IVAR_SHIFT 16
>
> and the above could be changed to:
>
> enum pci_device_ivars {
> PCI_IVAR_SUBVENDOR = IVAR_PCI << IVAR_SHIFT,
> PCI_IVAR_SUBDEVICE,
> PCI_IVAR_VENDOR,
>
> };
>
> and then we'd have an unambiguous key, and the bus could easily implement
> multiple interfaces.
>
> but then again, most of the existing interfaces in the kernel are mutually
> exclusive, so you could implement this just for your new interfaces.
>
ok, I think I got it now. You and I are exactly saying the same thing,
just in different terms; there is no way to currently specify multiple
independent (/overlapping) ivars in a child...
- Arnaud
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 12:37 AM, Warner Losh wrote: > > On Jul 8, 2012, at 9:46 PM, Arnaud Lacombe wrote: > >> On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh wrote: >>> >>> On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote: >>> >>>> Hi, >>>> >>>> On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh wrote: >>>>> >>>>> On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote: >>>>>> Ok, yet another Newbus' limitation. Assuming a device exports more >>>>>> than one interface, and one of its child has need to use more than one >>>>>> interface, each interfaces cannot register, concurrently, its own >>>>>> ivar. While I try to always have a single child per >>>>>> interface/resource, I need to keep some compatibility with the old way >>>>>> of doing thing (POLA wrt. drivers I cannot/will not convert and >>>>>> userland). So, it would have been nice if ivar had been per-interface, >>>>>> not global and unique to one device. >>>>> >>>>> There's one pointer for the ivars. The bus code gets to determine what >>>>> the ivar looks like, because the interface is totally private to the bus. >>>>> So long as it returns the right thing for any key that's presented, it >>>>> doesn't matter quite how things are done. >>>>> >>>>> So I'm not sure I understand what you are saying here. >>>>> >>>> dev0 implements two interfaces: A and B. dev1, child of dev0, needs to >>>> use both interfaces. There is no generic way for dev0 to export >>>> independent ivars for both interface. For now, I restricted the >>>> function of the second interface not to need ivar, but that's kind of >>>> hackish. >>> >>> Only if the IVARs for interface A and interface B have overlapping values. >>> If the Ivar keys don't overlap, then there's no problems at all. Certainly >>> less hackish than not using them at all. Since dev0 knows the layout of >>> the ivar that it set on its child, this presents no problems at all. It >>> would return the values from A from the right part of the ivar, and those >>> from B in the right part. Apart from the coordination of Ivar numbers, as >>> I outlined in my last post, there's no issue here. >>> >> I think we should not be talking about the same API here. I have no >> idea what you mean by "the key to value translation", nor "Ivar >> numbers". What I refer to is that device_set_ivars() / >> device_get_ivars() acts on a single instance variables from `struct >> device': `ivars'. In that case, I do not really see how to set that >> specific field to two distinct values for each interfaces. > > We are talking about the ivar interface. You are just misunderstanding how > it is used. > yes I indeed did... silly, silly me :-) - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 1:17 AM, Arnaud Lacombe wrote: > Hi, > > On Mon, Jul 9, 2012 at 12:37 AM, Warner Losh wrote: >> >> On Jul 8, 2012, at 9:46 PM, Arnaud Lacombe wrote: >> >>> On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh wrote: >>>> >>>> On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote: >>>> >>>>> Hi, >>>>> >>>>> On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh wrote: >>>>>> >>>>>> On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote: >>>>>>> Ok, yet another Newbus' limitation. Assuming a device exports more >>>>>>> than one interface, and one of its child has need to use more than one >>>>>>> interface, each interfaces cannot register, concurrently, its own >>>>>>> ivar. While I try to always have a single child per >>>>>>> interface/resource, I need to keep some compatibility with the old way >>>>>>> of doing thing (POLA wrt. drivers I cannot/will not convert and >>>>>>> userland). So, it would have been nice if ivar had been per-interface, >>>>>>> not global and unique to one device. >>>>>> >>>>>> There's one pointer for the ivars. The bus code gets to determine what >>>>>> the ivar looks like, because the interface is totally private to the >>>>>> bus. So long as it returns the right thing for any key that's >>>>>> presented, it doesn't matter quite how things are done. >>>>>> >>>>>> So I'm not sure I understand what you are saying here. >>>>>> >>>>> dev0 implements two interfaces: A and B. dev1, child of dev0, needs to >>>>> use both interfaces. There is no generic way for dev0 to export >>>>> independent ivars for both interface. For now, I restricted the >>>>> function of the second interface not to need ivar, but that's kind of >>>>> hackish. >>>> >>>> Only if the IVARs for interface A and interface B have overlapping values. >>>> If the Ivar keys don't overlap, then there's no problems at all. >>>> Certainly less hackish than not using them at all. Since dev0 knows the >>>> layout of the ivar that it set on its child, this presents no problems at >>>> all. It would return the values from A from the right part of the ivar, >>>> and those from B in the right part. Apart from the coordination of Ivar >>>> numbers, as I outlined in my last post, there's no issue here. >>>> >>> I think we should not be talking about the same API here. I have no >>> idea what you mean by "the key to value translation", nor "Ivar >>> numbers". What I refer to is that device_set_ivars() / >>> device_get_ivars() acts on a single instance variables from `struct >>> device': `ivars'. In that case, I do not really see how to set that >>> specific field to two distinct values for each interfaces. >> >> We are talking about the ivar interface. You are just misunderstanding how >> it is used. >> > yes I indeed did... silly, silly me :-) > Actually, no. I wasn't that silly, neither was I misunderstanding anything beside how *you* wanted it to be used, which is, I sorry to say, unacceptable. The last thing I want is to pollute an interface with a single-purpose, hand-crafted, bus. I was to just throw away all that ivar stuff and go into hinted child configuration for now, waiting for FDT... but of course, I figured out after a few hours that hinted child attachment requires `bus_hinted_child' to be set in the parent, as does bus_enumerate_hinted_children() / bus_generic_attach() to explicitly pollute my code. All this stuff should be done implicitly to support N:1 interfaces/client relationship. N *independent* interfaces being provided by a single driver; of course, I'm not even going back to require those interface being provided by multiple drivers, it is already a dead end. I am not even sure any driver in the tree provides more than one interface... For whatever reason, I am more and more thinking that this all new-bus[0] stuff is *way* overkill, static, bloated at will, and missing critical features; a huge PITA to use, for the intended purpose. /me pissed. - Arnaud [0]: damn, why is it even called "newbus", this stuff is 14 years old. It really belongs to a museum, not production code... ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 11:27 AM, John Baldwin wrote: > Also, I think we should do this in general. We already have one example (e.g. > ACPI IVARs start at 100 so that things like the ACPI PCI bus driver can > provide both ACPI and PCI IVARs to child devices). I think we should assign > each bus it's own IVAR range. It would make it easier to determine if a > specific IVAR is event supported. Certainly I think ISA and PCI at a minimum > should be changed to start at, say, 200 and 300. > What's the point doing that ? Let's just resign to the conclusion that FreeBSD devices' subsystem provides no dynamic way for a client device to deals with multiple bus driver. Instead all possible combination have to be harcoded and hand-crafted, when at all possible, to look like they're coming from a single bus... But again, newbus is 14 years old... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh wrote: > I'm sorry you feel that way. > > Honestly, though, I think you'll be more pissed when you find out that the > N:1 interface that you want is being done in the wrong domain. But I've been > wrong before and look forward to seeing your replacement. > > acpi_pcib_acpi.c, btw, implements both PCIB interfaces and ACPI interfaces. > Does it ? From the definition of `acpi_pcib_acpi_methods', I can only see a single pcib(4) interface being exported. Moreover, I do not seem to be able to find any clue that would led any ACPI devices to attach on acpi_pcib_acpi(4), neither to find how could acpi_get_flags() ends up in acpi_pcib_read_ivar() ? Thks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: CVE-2012-0217 Intel's sysret Kernel Privilege Escalation and FreeBSD 6.2/6.3
Hi, On Fri, Jul 13, 2012 at 11:02 AM, John Baldwin wrote: > On Friday, July 13, 2012 10:42:04 am Poul-Henning Kamp wrote: >> In message <201207130831.59211@freebsd.org>, John Baldwin writes: >> >> >Every FreeBSD/amd64 kernel in existent is vulnerable. In truth, my > personal >> >opinion is that Intel screwed up their implementation of that instruction >> >whereas AMD got it right, and we are merely working around Intel's CPU bug. > :( >> >> Given that the instruction set of AMD64 is defined by AMD originally, >> while Intel was trying very hard to ram Itanic down everybodys >> throat, that diagnosis is a given: Intel copied AMD, and difference >> in functionality is a screwup on Intels part, even if they documented >> their screwup in their manual. >> >> TL;DR: Which part of "compatible" doesn't Intel get ? > > In this case, I believe they were just lazy and reused some existing block to > manage this exception case without properly thinking through the security > implications of using a user-supplied stack pointer to handle a fault. > Just as FreeBSD's developers were lazy when new-bus was designed ? Honestly, what's the point of this rock throwing and ad-hominem attacks ? I could start throwing a few more CVE-2009-2936 or CVE-2009-4488; just to point out nobody's perfect... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh wrote: > [..] > Honestly, though, I think you'll be more pissed when you find out that the > N:1 interface that you want is being done in the wrong domain. But I've been > wrong before and look forward to seeing your replacement. > I will just pass function pointers for now, if things should be done dirty, let's be explicit about it. Now, the hinted device attachment did work quite smoothly, however, I would have a few suggestion: 1) add a call to bus_enumerate_hinted_children() before the call DEVICE_IDENTIFY() call in bus_generic_driver_added() this is required to be able to support dynamic loading and attachment of hinted children. 2) have a generic bus_hinted_child method which would just add a new child to the bus. 3) have bus_enumerate_hinted_children() and bus_generic_attach() always ran on device attachment. There is current +100 explicit call to bus_generic_attach() in the sys/dev/ tree. This should be done always and implicitly. 4) have bus_generic_detach() always ran prior to device detachment If not already the case. There is still the same +100 direct call to bus_generic_detach is the tree. 5) have the bus_generic_* method be the default of their respective method 6) have device_delete_child() called upon device detachment. As a rule of thumb, when a kld is unloaded there should not be any remains of anything built previously. Without device_delete_child() or proper singleton implementation, multiple load/unload sequence of bus will attempt to attach multiple version of a child, even if the single child was added prior to the bus_generic_attach() call. Also, as a rule of thumb, if the same logic is implemented in more than a few buses, it should be made generic and implicit. I am lazy, I hate doing the same things over and over, not to say it raised the likelihood of bugs' introduction... - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Fri, Jul 13, 2012 at 1:56 PM, Arnaud Lacombe wrote: > Hi, > > On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh wrote: >> [..] >> Honestly, though, I think you'll be more pissed when you find out that the >> N:1 interface that you want is being done in the wrong domain. But I've >> been wrong before and look forward to seeing your replacement. >> > I will just pass function pointers for now, if things should be done > dirty, let's be explicit about it. > > Now, the hinted device attachment did work quite smoothly, however, I > would have a few suggestion: > 1) add a call to bus_enumerate_hinted_children() before the call > DEVICE_IDENTIFY() call in bus_generic_driver_added() > > this is required to be able to support dynamic loading and attachment > of hinted children. > > 2) have a generic bus_hinted_child method which would just add a new > child to the bus. > > 3) have bus_enumerate_hinted_children() and bus_generic_attach() > always ran on device attachment. > > There is current +100 explicit call to bus_generic_attach() in the > sys/dev/ tree. This should be done always and implicitly. > > 4) have bus_generic_detach() always ran prior to device detachment > > If not already the case. There is still the same +100 direct call to > bus_generic_detach is the tree. > > 5) have the bus_generic_* method be the default of their respective method > > 6) have device_delete_child() called upon device detachment. > > As a rule of thumb, when a kld is unloaded there should not be any > remains of anything built previously. Without device_delete_child() or > proper singleton implementation, multiple load/unload sequence of bus > will attempt to attach multiple version of a child, even if the single > child was added prior to the bus_generic_attach() call. > > Also, as a rule of thumb, if the same logic is implemented in more > than a few buses, it should be made generic and implicit. > > I am lazy, I hate doing the same things over and over, not to say it > raised the likelihood of bugs' introduction... > could I at least get some feedback on the proposals above ? Thanks, - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Generic queue's KPI to manipulate mbuf's queue
Hi, AFAIK, there is no proper KPI for managing mbuf queue. All users have to re-implements the queue logic from scratch, which is less than optimal. From a preeminent FreeBSD developer at BSDCan 2009: "we do not need a new list implementation". There has been a few attempt of providing a queue API, namely , but that is nothing more than an ad-hoc solution to something which _has_to_be_ generic. For the sake of adding more mess in the tree, this implementation has been duplicated in ... Now, I understand, or at least merely witness without power, the reluctance of kernel hackers to have 'struct mbuf` evolves, especially wrt. their desire to keep binary compatibility of KPI[0]. Now, none of the current ad-hoc API matched my needs, and I really did NOT want to re-implement a new list implementation for missing basic operation, such as deleting an element of the list, so I came with the attached patch. The main idea is to be able to use already existing code from for mbuf queuing management. It is not the best which can be done. I am not a huge fan of keeping `m_nextpkt' and introducing a `m_nextelm', I would have preferred to use TAILQs, and I do not like the dwelling in SLIST internal implementation details. However, this change is relatively lightweight, and change neither ABI or API. Any comment appreciated. - Arnaud [0]: taking care of having a stable kernel ABI and *not* a stable userland ABI is beyond my understanding, but this is not the subject of this mail. mbuf_slist.diff Description: Binary data ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Generic queue's KPI to manipulate mbuf's queue
Hi, On Wed, Jul 25, 2012 at 7:25 AM, Andre Oppermann wrote: > On 24.07.2012 20:18, Arnaud Lacombe wrote: >> >> Hi, >> >> AFAIK, there is no proper KPI for managing mbuf queue. All users have > Before we can talk about an mbuf queue you have to define what you > want to "queue". Is it packets or an mbuf chain which doesn't have > clear delimiters (as with tcp for example)? Depending on that the > requirements and solutions may be vastly different. > I was thinking about queues as in "general use-case of m_nextpkt", that would be dummynet queuing, QoS, various reassembly queues, socket buffer, etc... >> to re-implements the queue logic from scratch, which is less than >> optimal. From a preeminent FreeBSD developer at BSDCan 2009: "we do >> not need a new list implementation". There has been a few attempt of >> providing a queue API, namely , but that is >> nothing more than an ad-hoc solution to something which _has_to_be_ >> generic. For the sake of adding more mess in the tree, this >> implementation has been duplicated in ... > > Duplication is always a sign for the need of a generic approach/KPI. > > >> Now, I understand, or at least merely witness without power, the >> reluctance of kernel hackers to have 'struct mbuf` evolves, especially >> wrt. their desire to keep binary compatibility of KPI[0]. Now, none of >> the current ad-hoc API matched my needs, and I really did NOT want to >> re-implement a new list implementation for missing basic operation, >> such as deleting an element of the list, so I came with the attached >> patch. The main idea is to be able to use already existing code from >> for mbuf queuing management. It is not the best which >> can be done. I am not a huge fan of keeping `m_nextpkt' and >> introducing a `m_nextelm', I would have preferred to use TAILQs, and I >> do not like the dwelling in SLIST internal implementation details. >> However, this change is relatively lightweight, and change neither ABI >> or API. > > IMO your change is a rather elegant way of introducing the LIST macros > to the mbuf nextpkt field. I do like it and don't object to it providing > you sufficiently answer the question in the first paragraph. > actually, I made a mistake selecting SLISTs, it should really be an STAILQ. It has the same advantage wrt. ABI, and most usage made of `m_nextpkt' follows a tail queue logic. The only advantage of TAILQ would be reverse traversal, and time constant removal of inner elements. - Arnaud > -- > Andre > >> Any comment appreciated. >> >> - Arnaud >> >> [0]: taking care of having a stable kernel ABI and *not* a stable >> userland ABI is beyond my understanding, but this is not the subject >> of this mail. >> >> >> >> ___ >> freebsd-...@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" >> > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi,
On Mon, Jul 30, 2012 at 5:06 PM, John Baldwin wrote:
> On Tuesday, July 17, 2012 2:03:14 am Arnaud Lacombe wrote:
>> Hi,
>>
>> On Fri, Jul 13, 2012 at 1:56 PM, Arnaud Lacombe wrote:
>> > Hi,
>> >
>> > On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh wrote:
>> >> [..]
>> >> Honestly, though, I think you'll be more pissed when you find out that
> the N:1 interface that you want is being done in the wrong domain. But I've
> been wrong before and look forward to seeing your replacement.
>> >>
>> > I will just pass function pointers for now, if things should be done
>> > dirty, let's be explicit about it.
>> >
>> > Now, the hinted device attachment did work quite smoothly, however, I
>> > would have a few suggestion:
>> > 1) add a call to bus_enumerate_hinted_children() before the call
>> > DEVICE_IDENTIFY() call in bus_generic_driver_added()
>> >
>> > this is required to be able to support dynamic loading and attachment
>> > of hinted children.
>
> I'm not sure this is a feature we want to support (to date hinted children
> have only been created at boot time).
>
>> > 2) have a generic bus_hinted_child method which would just add a new
>> > child to the bus.
>
> Possibly, but hinted children are intentionally opt-in and not enabled
> by default.
>
>> > 3) have bus_enumerate_hinted_children() and bus_generic_attach()
>> > always ran on device attachment.
>> >
>> > There is current +100 explicit call to bus_generic_attach() in the
>> > sys/dev/ tree. This should be done always and implicitly.
>
> No. One of the problems is that different busses want to do it at
> different times. It is usually done last, but some buses may want to
> do additional work after the bus_generic_attach().
>
>> > 4) have bus_generic_detach() always ran prior to device detachment
>
> Similar.
>
>> > 5) have the bus_generic_* method be the default of their respective
> method
>
> No. However, what would be a good idea (and one thing I've had on my
> list), is to create a "bus_generic" driver that uses the bus_generic_*
> methods for all it's methods and let bus drivers inherit from that to
> get the generic methods if they are appropriate. If you do this, I
> would also add a second "bus_generic_rl" that inherits from "bus_generic"
> but uses the generic resource list methods for resources.
>
>> > 6) have device_delete_child() called upon device detachment.
>
> No. This is for a bus to decide. This would be horrifically wrong
> for things like kldunloading a PCI device driver. Just because you
> unload a driver (so that it detaches from devices) does not mean those
> physical devices have gone away.
>
>> > As a rule of thumb, when a kld is unloaded there should not be any
>> > remains of anything built previously. Without device_delete_child() or
>> > proper singleton implementation, multiple load/unload sequence of bus
>> > will attempt to attach multiple version of a child, even if the single
>> > child was added prior to the bus_generic_attach() call.
>
> Hinted devices should perhaps be removed, yes. However, what other drivers
> often do is use a singleton approach instead (despite your claim that they
> don't).
>
FreeBSD's newbus device framework already sucks (as in "too
static"/"inflexible"), making it sucks even more (as in "more
static"/"more inflexible") might not be the wisest approach... This is
no longer the 90'. The good old enumerating-buses, tree-based, model
is to be relegated to a corner-case of the computer world with
profusion of embedded, non-enumerating, highly integrated, highly
interconnected, highly custom SoCs.
I am yet to see a robust approach to the various problem I submitted.
> For example:
> static void
> ipmi_isa_identify(driver_t *driver, device_t parent)
> {
> struct ipmi_get_info info;
> uint32_t devid;
>
> if (ipmi_smbios_identify(&info) && info.iface_type != SSIF_MODE &&
> device_find_child(parent, "ipmi", -1) == NULL) {
> ...
> BUS_ADD_CHILD(parent, 0, "ipmi", -1);
> }
> }
>
duplicated code doing the exact same abstract, hardcoded, function,
all over the tree, absolutely unacceptable, if not a blatant proof of
failure of what should be made generic, if not fully dynamic.
- Arnaud
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 30, 2012 at 11:51 PM, Warner Losh wrote: > [...] We lack that right now, which is why you're trying to shoe-horn the FDT > connections into a newbus world and complaining that everything sucks because > it is a poor fit. I'd suggest that different mechanisms are necessary. > I'm not trying anything, or at least no longer. I do not see the point working on that when I can not get trivial patches in, especially those fixing poorly maintained drivers, whose issues _do_ affect people. - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: newbus' ivar's limitation..
Hi, On Tue, Jul 31, 2012 at 12:27 PM, Warner Losh wrote: > > On Jul 31, 2012, at 9:20 AM, Arnaud Lacombe wrote: > >> Hi, >> >> On Mon, Jul 30, 2012 at 11:51 PM, Warner Losh wrote: >>> [...] We lack that right now, which is why you're trying to shoe-horn the >>> FDT connections into a newbus world and complaining that everything sucks >>> because it is a poor fit. I'd suggest that different mechanisms are >>> necessary. >>> >> I'm not trying anything, or at least no longer. I do not see the point >> working on that when I can not get trivial patches in, especially >> those fixing poorly maintained drivers, whose issues _do_ affect >> people. > > Hey Arnaud, sorry to be a little harsh, but maybe if you shouted less and > cooperated more, people would be more willing to work with you. > I tried to be Mr Nice Guy in the past, all I got in return was being ignored. Lists are full of unanswered problem because people do not want to insist getting an answer. Now, believe me on one point, if you are a driver or subsystem author, might I have an issue with your work, I *will* be a recurring pain in your butt to get the issue fixed, or to get in what I do believe, with the limited set of knowledge and resources to my disposal[0], to be a correct fix for the issue, at the time. If you are condescending, arrogant, or advocates of the status-quo, as have been committers in the past, I will return you favor. Let face it, FreeBSD is not the most outstanding OS out there (despite obvious capabilities), and I would not be too proud of its state. All that to say that asking politely does not work in the arbitrary FreeBSD realm, where "the power to serve", is today nothing more that a relic. - Arnaud ps: note that I am ready to be wrong and to be publicly proven so, as it's been the case recently. [0]: which certainly not mean I am wrong, very far from that. > Having said that, I'd be happy to help fix this problem. I've not seen the > patches you're talking about, I'd be happy to take a look and try to get them > in, as appropriate. > > Warner > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao wrote: > > You don't want to work cooperatively. > Why is it that mbuf's refactoring consultation is being held in internal, private, committers-and-invite-only-restricted meeting at BSDCan ? Why is it that so much review and discussion on changes are held privately ? - Arnaud ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 12:40 PM, Attilio Rao wrote: > On Wed, Aug 1, 2012 at 5:32 PM, Arnaud Lacombe wrote: >> Hi, >> >> On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao wrote: >>> >>> You don't want to work cooperatively. >>> >> Why is it that mbuf's refactoring consultation is being held in >> internal, private, committers-and-invite-only-restricted meeting at >> BSDCan ? >> >> Why is it that so much review and discussion on changes are held privately ? > > Arnaud, > belive me, to date I don't recall a single major technical decision > that has been settled exclusively in private (not subjected to peer > review) and in particular in person (e-mail help you focus on a lot of > different details that you may not have under control when talking to > people, etc). > Whose call is it to declare something worth public discussion ? No one. Every time I see a "Suggested by:", "Submitted by:", "Reported by:", and especially "Approved by:", there should to be a public reference of the mentioned communication. > Sometimes it is useful that a limited number of developers is involved > in initial brainstorming of some works, > Never. > but after this period > constructive people usually ask for peer review publishing their plans > on the mailing lists or other media. > Again, never. By doing so, you merely put the community in a situation where, well, "We, committers, have come with this, you can either accept or STFU, but no major changes will be made because we decided so." The callout-ng conference at BSDCan was just beautiful, it was basically: Speaker: "we will do this" Audience: "how about this situation ? What you will do will not work." Speaker: "thank you for listening, end of the conference" It was beautiful to witness. > If you don't see any public further discussion this may be meaning: > a) the BSDCan meetings have been fruitless and there is no precise > plan/roadmap/etc. > so not only you make it private, but it shamelessly failed... > b) there is still not consensus on details > Then the discussion should stop, public records are kept for reference in the future. There is no problem with this. > and you can always publically asked on what was decided and what not. > Just send a mail to interested recipients and CC any FreeBSD mailing > list. > This is not the way "openness" should be about. - Arnaud > Attilio > > > -- > Peace can only be achieved by understanding - A. Einstein ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 2:18 PM, Attilio Rao wrote: > On 8/1/12, Arnaud Lacombe wrote: >> Hi, >> >> On Wed, Aug 1, 2012 at 12:40 PM, Attilio Rao wrote: >>> On Wed, Aug 1, 2012 at 5:32 PM, Arnaud Lacombe >>> wrote: >>>> Hi, >>>> >>>> On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao >>>> wrote: >>>>> >>>>> You don't want to work cooperatively. >>>>> >>>> Why is it that mbuf's refactoring consultation is being held in >>>> internal, private, committers-and-invite-only-restricted meeting at >>>> BSDCan ? >>>> >>>> Why is it that so much review and discussion on changes are held >>>> privately ? >>> >>> Arnaud, >>> belive me, to date I don't recall a single major technical decision >>> that has been settled exclusively in private (not subjected to peer >>> review) and in particular in person (e-mail help you focus on a lot of >>> different details that you may not have under control when talking to >>> people, etc). >>> >> Whose call is it to declare something worth public discussion ? No one. >> >> Every time I see a "Suggested by:", "Submitted by:", "Reported by:", >> and especially "Approved by:", there should to be a public reference >> of the mentioned communication. > > Not necessarilly. Every developer must ensure to produce a quality > work, with definition of "quality" being discretional. Some people > fail this expectation, while others do very good. As a general rule, > some people send patches to experts on the matter and they just trust > their judgment, others also full-fill testing cycles by thirdy part > people, others again ask for public reviews. Often this process is > adapted based on the dimension of the work and the number of > architectural changes it introduces. > > As a personal matter, for a big architectural change things I would > *personally* do are: > - Prepare a master-plan with experts of the matter > - Post a plan (after having achived consensus) on the public mailing > list for further discussions > - Adjust the plan based on public feedbacks (if necessary) > - Implement the plan > - Ask the experts if they have objections to the implementation > - Ask testers to perform some stress-testing > - Ask testers to perform benchmark (if I find people interested in that) > - Send out to the public mailing list for public review > - Integrate suggestions > - Ask testers to stress-test again > - Commit > > I think this model in general works fairly well, > I agree. > but people may have > different ideas on that, meaning that people may want to not involve > thirdy part for testing or review. This is going to be risky and lower > the quality of their work but it is their call. > >>> Sometimes it is useful that a limited number of developers is involved >>> in initial brainstorming of some works, >>> >> Never. >> >>> but after this period >>> constructive people usually ask for peer review publishing their plans >>> on the mailing lists or other media. >>> >> Again, never. By doing so, you merely put the community in a situation >> where, well, "We, committers, have come with this, you can either >> accept or STFU, but no major changes will be made because we decided >> so." > > You are forgetting one specific detail: you can always review a work > *after* it entered the tree. This is something you would never do, but > sometimes, when poor quality code is committed there is nothing else > you can do than just raise your concern after it is in. > Unfortunately, not. First, the developer will certainly have moved on after the commit, API may have been changed tree-wide, etc. Then, time is likely to have passed between the time you figure potential regression or bugs, which makes the first point even more problematic. Finally, if my point of view is being ignored *before* it goes to the tree, do you really expect it will be considered *after* ? >From my external point of view, committers not only have the possibility, but *do* commit mess in the tree without non-committers could say or do anything, just as well as committers being able to arbitrarily close PR even if the original reporter disagree. >> The callout-ng conference at BSDCan was just beautiful, it was basically: >> >> Speaker: "we will do this" >> Audience: "how about this situation ? What you will do will not work." >> Speaker: "thank you for listening, end of the conference&quo
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 4:06 PM, Adrian Chadd wrote: > Any interested party is very welcome to approach a developer and get > added to the developer summits. Plenty of the people at the most > recent developer summit weren't @freebsd.org committers - we had > plenty of representation from companies using FreeBSD. > > If you want to participate, just ask a friendly developer who is going > to the developer summit to sponsor you in going. You're pleasant in > person, so I'd have no problem sponsoring you if I am going to an > event. :) > I have a very deep, quasi-philosophical, trouble/problem with that whole idea of sponsor-requirement to attend a such meeting. There is just something which does not feel right about it. From my point of view, this is a matter of common sense, focus is gonna be very narrow and deeply technical. Attendee should go there only if they think they will give positive feedback. As for myself, I would not attend a developer meeting on the fiber-channel over infiniband optimization, but would attend a developer meeting on next-generation mbuf. Now, maybe I'll just push the door of some developer meeting I'd be interested in during next BSDCan, and see what happen :-) The outcome might be interesting to study in a social interaction, prisoner dilemma related, point-of-view. - Arnaud > And/or, work with warner to get improvements into the tree and someone > will sponsor a commit bit for you. > > Perhaps we as developers should more openly publish the results of > developer summits. But as I said, they're not "closed" - they're just > "invite only for non-developers." We're not going to exclude anyone > from coming unless they really ARE going to just sit there and troll. > You're motivated, you're enthusiastic and you want to see things > change for the better. You're also not confrontational in person. I > have no problem with you coming along. > > > Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 7:28 PM, Warner Losh wrote: > > On Aug 1, 2012, at 3:39 PM, Arnaud Lacombe wrote: > >> Hi, >> >> On Wed, Aug 1, 2012 at 4:06 PM, Adrian Chadd wrote: >>> Any interested party is very welcome to approach a developer and get >>> added to the developer summits. Plenty of the people at the most >>> recent developer summit weren't @freebsd.org committers - we had >>> plenty of representation from companies using FreeBSD. >>> >>> If you want to participate, just ask a friendly developer who is going >>> to the developer summit to sponsor you in going. You're pleasant in >>> person, so I'd have no problem sponsoring you if I am going to an >>> event. :) >>> >> I have a very deep, quasi-philosophical, trouble/problem with that >> whole idea of sponsor-requirement to attend a such meeting. There is >> just something which does not feel right about it. From my point of >> view, this is a matter of common sense, focus is gonna be very narrow >> and deeply technical. Attendee should go there only if they think they >> will give positive feedback. As for myself, I would not attend a >> developer meeting on the fiber-channel over infiniband optimization, >> but would attend a developer meeting on next-generation mbuf. >> >> Now, maybe I'll just push the door of some developer meeting I'd be >> interested in during next BSDCan, and see what happen :-) The outcome >> might be interesting to study in a social interaction, prisoner >> dilemma related, point-of-view. > > Given how ridiculously easy it is to get a proper invite, there's not need to > be a jerk just to prove an obscure philosophical point about attendance. > There's plenty of time to do that over the technical points being discussed. > Let me explain my thoughts: I do not recognize the committers legitimacy to give such invite, and to some extend, I do not recognize committers self-given legitimacy altogether. This do not mean I'd praised a structure-less project; quite the opposite actually. Starting from that, I will certainly not defer to anybody to request such invite or commit bit. Feel free to kick me out of the meeting room if you want to; I would have proved my point. Now, if invites are so easy to get, just get rid of it. It's a worthless, cumbersome item. - Arnaud ps: please, do not get me wrong, I would apply this policy to anybody who propose to help. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
