RE: Slowing down an old program to run on a fast CPU?
You could try installing vmware and running however many copies of windows it takes to make the game playable... (i would say some other form of *BSD, but it probobly wouldn't hog as much cpu :P) ~NVX Or try qemu. I yesterday booted installed NetBSD in a qemu box running under FreeBSD5.4 ;-) Try to run it with/without 'kldload kqemu'. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: usbd.conf: detach ukbd
Else if devd is not available on 4.11 you will have to change
some code and
compile a new kernel, from what I can see.
To the file /sys/dev/usb/ukbd.c add this:
static void
usbd_add_device_detach_event(device_t self)
{
struct usb_event ue;
bzero(ue, sizeof(ue));
strlcpy(ue.u.ue_device.udi_devnames[0],
device_get_nameunit(self), USB_MAX_DEVNAMELEN) ;
usb_add_event(USB_EVENT_DEVICE_DETACH, ue);
return;
}
ukbd_detach()
{
...
usbd_add_device_detach_event(self);
return (0);
}
This will make the suggestion from Maksim work.
Ok, that seems to work with a minor change [no strlcpy]
and two additional patches in usb.h usb.c.
Usbd gets a detach event ukbd0 and another event from its fallthrough
device.
Thank you once again.
If someone is interested in the patch files against 4.11 see the attachment.
Norbert
ukbd.c.patch
Description: Binary data
usb.c.patch
Description: Binary data
usb.h.patch
Description: Binary data
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
FreeBSD 5.3/4 vs 4.11
Hello list, I've recently been doing quite a few benchmarks with regard to PHP and Apache, as well as stripping down the Kernel on 5.3 and 5.4 to improve performance for web/application servers. My question is: I was wondering if anyone out there has ever done a head to head test of 4.11 to 5.3 (or 5.4) and if so, is 4.11 faster in any areas and/or does it provide any benefits over 5.3+? Thanks. Ray ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: using vkbd device
you also might want to look at experimental keyboard mux drivers. it is based on vkbd(4) and uses the idea of one super-keyboard that consumes scancodes from other keyboards. there are still a few issues i need to fix, but, in general, it works. http://www.geocities.com/m_evmenkin/kbdmux-2.tar.gz thanks, max Hello Max, I back-ported kbdmux to FreeBSD 4.11 (and stopped work on vkbd). It seems to work, although I still have some stability issues with dynamically attaching/detaching a usb keyboard. For your reference I have enclosed a patch file. Sorry for some diffs due to slightly different coding and debugging style. Thank you for any comments, Norbert kbdmux.c.patch Description: Binary data dev_debug.h Description: Binary data ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
contigmalloc() and mmap()
I have some experiences in writing Linux device driver, but I am new to FreeBSD kernel, although from the first glimpse, there seems no big differences between the kernel operations a char device driver can use, but I met some problems when the driver is running in FreeBSD. Our device is an experimental communication card, which can do remote DMA between two cards, which means the card in one node can DMA memory data to network, and when data are transfered to the card on another node, it will be DMAed to memory too. Because the card can only use contiguous physical memory for DMA operations, so data in user process will be copied to an contiguous memory buffer, then DMA will tranfer data in this buffer, and this buffer is allocated by driver using __get_free_pages() in Linux. The buffer is mmaped to user process space, so user process can do the copy directly in user space, it can directly orgnize data in this mmaped memory too. When I am porting my driver to FreeBSD, I use /dev/bktr driver as the example, seems easily, just using contigmalloc() to allocate the buffer, and in driver's _mmap() function, return the physical address for each page to be mmaped. The problem is, in Linux, I allocate buffer in driver's ioctl() function, and free it in a timer function, many processes may use the driver at the same time, each process use a different kernel buffer, when the process first use the driver, it calls __get_free_pages() to allocate kernel buffer, and when it exit, it trigger timer function, the timer function will can free_pages() to free the buffer, so these two kernel interfaces will be called frequently, but this usage pattern works correctly in Linux. In FreeBSD, the driver works in the same pattern, but when a user process mmap driver's buffer (allocated by contigmalloc()) and is killed, then when another process mmap the same buffer again, sometimes it cannot get correct data from the mmaped pages, which means the user space virtual aderess may not point to the correct physical page of driver's buffer, sometimes the OS even panic with some information such as Trap 12, Page not present etc. I browsed kernel tree, I found those drivers which use contigmalloc() and contigfree() always call these two kernel interfaces in _attach() and _detach(), but in my driver, I call contigmalloc() in ioctl(), and call contigfree() in a callout function which is set by callout_reset(). What I want to know is if contigmalloc() and contigfree() can only be used under some conditions? And recently, I modified my driver, I allocated a big chunk of contiguous physical memory using contigmalloc() in the driver's _attach() function, and I use a simple first-fit algorithm to manage this memory myself, which mean in ioctl() I use my allocate/deallocate functions instead of contigmalloc(), in _detach() function contigfree() is called to free the big chunk of memory, no panic again, but sometimes, process cannot get the correct data from the mmaped memory. I don't know why. Any help is welcomed. Thanks. _ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: FreeBSD 5.3/4 vs 4.11
yeah, thanks. I'm subscribed to performance also - it's kinda dead over there. I'll check the archives also, thanks. Ray At 11:42 AM 6/13/2005 +0200, Julian H. Stacey wrote: | [EMAIL PROTECTED] wrote: | Hello list, | |I've recently been doing quite a few benchmarks with regard to PHP and Apache, | as well as stripping down the Kernel on 5.3 and 5.4 to improve performance for | web/application servers. | | My question is: I was wondering if anyone out there has ever done a head to head | test of 4.11 to 5.3 (or 5.4) and if so, is 4.11 faster in any areas and/or does | it provide any benefits over 5.3+? | | There was a discussion of performance just recently on one if the | main lists (ie current or hackers or stable, can't remember) However | you'll find it by browsing the the mail archives via | http://www.freebsd.org. | | FreeBSD has a mail list dedicated to discussing performance issues. | I suggest you subscribe it, after first browsing those archives: | http://lists.freebsd.org/pipermail/freebsd-performance/ | | - | Julian StaceyNet Sys Eng Consultant, Munich http://berklix.com | Mail in Ascii (Html=Spam). Ihr Rauch = mein allergischer Kopfschmerz. | | ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Tracking FreeBSD performance over time - what hackers want?
Hello Hackers!
I have an idea which could be used to track FreeBSD performance and regression
testing. Please take a look and give Your opinion on how usefull that would be
for the FreeBSD project.
The basis for this system would be hijacking certain functions execution with
injected code. Hijacker program runs target binary with LD_PRELOAD to inject
some code (code doing statistics, tests, etc..) and installs redirections using
ptrace (injecting jump instructions here and there). It then deattaches and the
program runs almost as normal. (A little demo of how this might look later).
This approach has its pros and cons:
+ no context switch to fire hijacking code
+ flexible - You can hijack any function that has a symbol. Hijacking PLT
(done) and relocations in shared libraries (will be done) included.
+ transparent - there needs to be no modiffication in target binary; hackers
don't own all the hardware. It's hard to ask people with interesting equipement
to recompile their binaries with profiling options. This would allow them to
measure performance without changing their system.
+ small performance impact - put code only in places You want, and put there
only the code which is needed there. (can be made O(1) unless You make it worse)
- ABI / Architecture dependent
- needs all the shared library mechanism to inject code (LD_PRELOAD)
- needs symbols
- writes the ro code pages on installing the redirections (negligible?)
And here is the demo of poc code (not particulary usefull..but..)
This hijack code has a static table indexed by read sizes in which we count how
many times a read of this size was called.
#include hijack.h
#include stdio.h
#define PROBES 400
#define QUANT 0x10
static int stats[PROBES];
static int max_read;
// RV(type) gives return value of the code
HIJACK(int, read, (int fd, char *buf, size_t size),
/* before call*/
,
/* after call*/
printf(read() = %d\n, RV(int));
if (max_read RV(int))
max_read = RV(int);
if ((RV(int) =0) (RV(int) QUANT * PROBES))
{
stats[RV(int) / QUANT]++;
}
)
HIJACK(int, main, (),
printf(hello hijacking!\n);
,
)
HIJACK(int, exit, (),
int i;
for (i=0; iPROBES; i++)
if (stats[i])
fprintf(stdout,%04d %04d\n, i*QUANT, stats[i]);
fprintf(stdout,and max read was %d\n, max_read);
fclose(stdout);
printf(finish!\n);
return RV(int);
,
)
--
i compile it
gcc -o apa.so -shared -I. apa.c hijack.c
(hijack.c has some required runtime functions)
I run it on apache:
./hijack -c ./apa.so -h *libc.so.5:read -h *libc.so*:exit -h main
/usr/local/sbin/httpd -X
hello
read() = 4096
read() = 4096
read() = 2732
(some of these)
read() = 0
read() = 961
read() = 425
read() = 0
(here i press ^C)
0010
0112 0002
0256 0002
0416 0003
0512 0002
0656 0002
0960 0001
1120 0002
2720 0002
4096 0046
and max read was 4096
So, again, please send Your opinions on this idea. Do You think this (+ all
needed utilities to do statistics etc) would be applicable to Summer Of Code
Tracking performance over time project?
best regards,
--
Marcin Koziej
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Tracking FreeBSD performance over time - what hackers want?
So, again, please send Your opinions on this idea. Do You think
this (+ all needed utilities to do statistics etc) would be
applicable to Summer Of Code Tracking performance over time project?
A very good (and simple) way to track and plot peformance over
time is Orca. You only need to generate simple text files with the
data, and Orca creates {beauti,use}ful graphs. Moreover, you can
generate the graphs and keep the rrd files from a different machine.
http://www.orcaware.com
I know, I know, I'm being slow with devilator :/ (I'm working on
a data collector for FreeBSD), but I'm stuck in an urgent project
right now :(
Borja.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
Hi,
I stumbled across this bug a year ago, but still none has managed to fix it.
Instead the PR got lost and I am now posting it a second time:
http://www.freebsd.org/cgi/query-pr.cgi?pr=80980
In FreeBSD 6-current the code for bus_space_write_multi_1() says:
__asm __volatile( \n\
cld \n\
1: lodsb \n\
movb %%al,(%2) \n\
loop 1b:
=S (addr), =c (count) :
r (bsh + offset), 0 (addr), 1 (count) :
%eax, memory, cc);
This is equivalent to:
while(--count)
{
/* I/O */
}
which is obviously wrong, because it doesn't check for count equal to zero. So
how can I fix this in assembly. I am not an expert with inlined assembly, so
maybe someone can correct me if I am wrong, but something like this needs to
be added:
or %ecx, %ecx
jz 2
2:
Another solution would be to wrap the inlined assembly into
if(count)
{
...
}
So can someone have this fixed, or is there a reason not to fix it. The one
who wrote the code has done the same mistake with every one of the
bus_space_ that does memory mapped I/O. It currently breaks my drivers.
--HPS
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
http://www.freebsd.org/cgi/query-pr.cgi?pr=80980
In FreeBSD 6-current the code for bus_space_write_multi_1() says:
__asm __volatile( \n\
cld \n\
1: lodsb \n\
movb %%al,(%2) \n\
loop 1b:
=S (addr), =c (count) :
r (bsh + offset), 0 (addr), 1 (count) :
%eax, memory, cc);
This is equivalent to:
while(--count)
{
/* I/O */
}
which is obviously wrong, because it doesn't check for count equal to zero.
So
how can I fix this in assembly. I am not an expert with inlined assembly,
so
maybe someone can correct me if I am wrong, but something like this needs
to
be added:
or %ecx, %ecx
jz 2
2:
This is wrong beacause the result is stored in ecx. Better using JECXZ
instruction
before the loop.
Greeting,
rookie
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Slowing down an old program to run on a fast CPU?
On Fri, Jun 10, 2005 at 04:57:50PM -0700, Matthew Dillon wrote: You think that is bad, try running 'rain' on an xterm! HAHA. Not only xterm, normal console is good enough :) Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
On Mon, Jun 13, 2005 at 02:12:38PM +0200, Hans Petter Selasky wrote:
This is equivalent to:
while(--count)
{
/* I/O */
}
which is obviously wrong, because it doesn't check for count equal to
zero.
Why do you think it is a bug? It is part of the interface contract and
useful to avoid an unnecessary check in 99% of the cases.
Joerg
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
So how can I fix this in assembly. I am not an expert with inlined assembly, so maybe someone can correct me if I am wrong, but something like this needs to be added: or %ecx, %ecx jz 2 2: This is wrong beacause the result is stored in ecx. Better using JECXZ instruction before the loop. Greeting, rookie No, it's a correct method to set/reset the zero flag: (X | X) == X just as (X X) == X So, he could also write: and %ecx, %ecx. I may be wrong, but in the old 386/486 days the jecxz was even less efficient, wasn't it? history Twenty years ago, my z80 programs had a lot of lines like and a ret z Weren't there discussions, if an nmos cpu consumed more electric power with either and a or or a? ;-) /history Norbert ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
From: Mark Tinguely [EMAIL PROTECTED] To: [EMAIL PROTECTED], freebsd-hackers@freebsd.org Subject: Re: contigmalloc() and mmap() Date: Mon, 13 Jun 2005 07:41:33 -0500 (CDT) I browsed kernel tree, I found those drivers which use contigmalloc() and contigfree() always call these two kernel interfaces in _attach() and _detach(), but in my driver, I call contigmalloc() in ioctl(), and call contigfree() in a callout function which is set by callout_reset(). What I want to know is if contigmalloc() and contigfree() can only be used under some conditions? Allocating early prevents unfullfilled requests due to fragmentation of the physical memory. I believe starting in FreeBSD 5.x, contigmalloc() started to fill physical memory requests from the higher memory locations to help with fragmentation. FreeBSD 5/6 has a Unified Meory Allocator that helps allocate/free cycles. And recently, I modified my driver, I allocated a big chunk of contiguous physical memory using contigmalloc() in the driver's _attach() function, and I use a simple first-fit algorithm to manage this memory myself, which mean in ioctl() I use my allocate/deallocate functions instead of contigmalloc(), in _detach() function contigfree() is called to free the big chunk of memory, no panic again, but sometimes, process cannot get the correct data from the mmaped memory. I don't know why. Are you sure that you are allocating page length request on page boundaries? Are you checking the success/failure of the allocation? Your page faults before implementing your own allocation sounds like you did not check the return status from contigmalloc() and were dereferencing a pointer on page 0. --Mark Tinguely. I checked the return values of those functions in my code, the problem I meet now seems caused by some cache coherency, I am still not sure, maybe I should do some more tests. And I used /dev/mem and bktr driver as the template when I was porting my driver, the platform I use is IA-64, in sys/ia64/ia64/mem.c, I find this line: *paddr = IA64_PHYS_TO_RR7(offset); why return the region 7 address? _ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
On Monday 13 June 2005 14:44, Joerg Sonnenberger wrote:
On Mon, Jun 13, 2005 at 02:12:38PM +0200, Hans Petter Selasky wrote:
This is equivalent to:
while(--count)
{
/* I/O */
}
which is obviously wrong, because it doesn't check for count equal to
zero.
Why do you think it is a bug? It is part of the interface contract and
useful to avoid an unnecessary check in 99% of the cases.
Where is it documented?
This is a bug, because it will break standard FIFO design. Consider the
following pseudo code:
static void
filter(struct fifo *f)
{
do {
if(!f-len)
{
if(f-m) ...;
f-m = get_mbuf();
if(!f-m) break;
f-len = m-m_len;
f-ptr = m-m_data;
}
/* f-Z_chip is the maximum transfer length */
io_len = min(f-len, f-Z_chip);
bus_space_write_multi_1(t,h,xxx,f-ptr,io_len);
f-len -= io_len;
f-Z_chip -= io_len;
f-ptr += io_len;
} while(Z_chip);
}
This code is going to crash if f-Z_chip or f-len is zero. That happens
when one is trying to send or receive a zero length frame or mbuf. So then
one can argue wether one should allow zero length frames or not. I argue that
zero length frames should be allowed, because it enables easy stream
synchronization:
For example to signal end of stream, send a frame less than 16 bytes. Maximum
frame length is 16 bytes. If a stream is exactly 16 bytes long, then one has
to send one 16 byte frame and one 0 byte frame.
Also zero length frames can be used as a kind of heart-beat.
Adding that extra check for zero transfer length is not going to affect
performance at all. If one does that using C, the compiler can optimize
away that if(count) ... when count is a constant. Besides the i386
machine instructions ins and outs already exhibit that behaviour.
--HPS
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: using vkbd device
Hi Norbert, you also might want to look at experimental keyboard mux drivers. it is based on vkbd(4) and uses the idea of one super-keyboard that consumes scancodes from other keyboards. there are still a few issues i need to fix, but, in general, it works. http://www.geocities.com/m_evmenkin/kbdmux-2.tar.gz I back-ported kbdmux to FreeBSD 4.11 (and stopped work on vkbd). It thanks for your work and patches! i will merge some of your changes into my version. this way it will be easier to maintain (that is if kbdmux(4) will be included in FreeBSD :-) seems to work, although I still have some stability issues with dynamically attaching/detaching a usb keyboard. For your reference I would you share with us what sort of issues you are having? feel free to move this into private email if you are not comfortable discussing it on public mailing list. have enclosed a patch file. Sorry for some diffs due to slightly different coding and debugging style. no problem. in one of your previous emails you have been concerned about possible keyboard interrupt miss (in vkbd(4)). kbdmux(4) have the same issue. is that something you just did not fix yet, or it is not a problem anymore? in my local tree a added an extra callout that goes 10 times a second and queues interrupt if needed. also i'm curious why do you use splhigh() instead of spltty()? is this an issue with usb? (not sure which spl level usb runs on in 4.x) thanks, max ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
possible problem writing to SCSI tapes
Hi Everyone, I thought I'd forward this, in case there is someone interested in working on SCSI tape support. Andrew Hume wrote in the June 2005 ;login: For outright bugs, two examples come to mind. The first is the weakness of the FreeBSD SCSI system; we cannot reliably write tapes on our FreeBSD nodes (although at least we get told about the errors!). Again, the tape is slow (5MB/s) and should not be an issue, and we can reliably write them on Linux (on more or less identical hardware). Although this is annoying, it turns out reading a tape works just fine, so we're not too annoyed. There's not further information in the article, nor is there a related PR that I can see. Perhaps someone interested can contact [EMAIL PROTECTED] directly. Cheers, -- Jacques A Vidrine / NTT/Verio [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Apache Xie wrote: I have some experiences in writing Linux device driver, but I am new to FreeBSD kernel, although from the first glimpse, there seems no big differences between the kernel operations a char device driver can use, but I met some problems when the driver is running in FreeBSD. Our device is an experimental communication card, which can do remote DMA between two cards, which means the card in one node can DMA memory data to network, and when data are transfered to the card on another node, it will be DMAed to memory too. sounds like a firewire card :-) Because the card can only use contiguous physical memory for DMA operations, so data in user process will be copied to an contiguous memory buffer, then DMA will tranfer data in this buffer, and this buffer is allocated by driver using __get_free_pages() in Linux. The buffer is mmaped to user process space, so user process can do the copy directly in user space, it can directly orgnize data in this mmaped memory too. When I am porting my driver to FreeBSD, I use /dev/bktr driver as the example, seems easily, just using contigmalloc() to allocate the buffer, and in driver's _mmap() function, return the physical address for each page to be mmaped. The problem is, in Linux, I allocate buffer in driver's ioctl() function, and free it in a timer function, many processes may use the driver at the same time, each process use a different kernel buffer, when the process first use the driver, it calls __get_free_pages() to allocate kernel buffer, and when it exit, it trigger timer function, the timer function will can free_pages() to free the buffer, so these two kernel interfaces will be called frequently, but this usage pattern works correctly in Linux. In FreeBSD, the driver works in the same pattern, but when a user process mmap driver's buffer (allocated by contigmalloc()) and is killed, then when another process mmap the same buffer again, sometimes it cannot get correct data from the mmaped pages, which means the user space virtual aderess may not point to the correct physical page of driver's buffer, sometimes the OS even panic with some information such as Trap 12, Page not present etc. I browsed kernel tree, I found those drivers which use contigmalloc() and contigfree() always call these two kernel interfaces in _attach() and _detach(), but in my driver, I call contigmalloc() in ioctl(), and call contigfree() in a callout function which is set by callout_reset(). What I want to know is if contigmalloc() and contigfree() can only be used under some conditions? Maybe I don't understand the problem but.. I think the problem is that you want to keep a separate buffer for each user, ] while the drivers you are looking at expect to have only one buffer per device. One answer to this would be to make each user open a different 'instance' of the device. (i.e. a differnt minor number). otherwise there is no really good place to store the information. The device does not track users as such and even if it did, how would it track when a user process forks and becomes 2? it is not notified of this event. And recently, I modified my driver, I allocated a big chunk of contiguous physical memory using contigmalloc() in the driver's _attach() function, and I use a simple first-fit algorithm to manage this memory myself, which mean in ioctl() I use my allocate/deallocate functions instead of contigmalloc(), in _detach() function contigfree() is called to free the big chunk of memory, no panic again, but sometimes, process cannot get the correct data from the mmaped memory. I don't know why. Any help is welcomed. Thanks. _ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, 13 Jun 2005, Julian Elischer wrote: Maybe I don't understand the problem but.. I think the problem is that you want to keep a separate buffer for each user, ] while the drivers you are looking at expect to have only one buffer per device. One answer to this would be to make each user open a different 'instance' of the device. (i.e. a differnt minor number). otherwise there is no really good place to store the information. The device does not track users as such and even if it did, how would it track when a user process forks and becomes 2? it is not notified of this event. Is he looking for something like this in FreeBSD? http://docs.sun.com/app/docs/doc/802-5900/6i9kj7or8?a=view -- DE ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Daniel Eischen wrote: On Mon, 13 Jun 2005, Julian Elischer wrote: Maybe I don't understand the problem but.. I think the problem is that you want to keep a separate buffer for each user, while the drivers you are looking at expect to have only one buffer per device. One answer to this would be to make each user open a different 'instance' of the device. (i.e. a differnt minor number). otherwise there is no really good place to store the information. The device does not track users as such and even if it did, how would it track when a user process forks and becomes 2? it is not notified of this event. Is he looking for something like this in FreeBSD? http://docs.sun.com/app/docs/doc/802-5900/6i9kj7or8?a=view Intersting, but no, I don't thionk that is what he is looking for. Several times in the past we've seen people complainign that Linux allows a device driver to know who called it and somehow it seems to store somewhere some information about who openned the device.. thos somehow allows linux to store an arbitrary structure for each openning process. I thin from the sond of it that he wants to do something similar. From the sond of it he wants to have a different buffer be used depending on who is calling. This would partly work but would not work when processes fork etc. I think Linux must do some extra housekeeping in this case. anyhow I may be wrong. I'll go read it again :-) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
2nd try to answer this.. Apache Xie wrote: I have some experiences in writing Linux device driver, but I am new to FreeBSD kernel, although from the first glimpse, there seems no big differences between the kernel operations a char device driver can use, but I met some problems when the driver is running in FreeBSD. Our device is an experimental communication card, which can do remote DMA between two cards, which means the card in one node can DMA memory data to network, and when data are transfered to the card on another node, it will be DMAed to memory too. Because the card can only use contiguous physical memory for DMA operations, so data in user process will be copied to an contiguous memory buffer, then DMA will tranfer data in this buffer, and this buffer is allocated by driver using __get_free_pages() in Linux. The buffer is mmaped to user process space, so user process can do the copy directly in user space, it can directly orgnize data in this mmaped memory too. ok, so far so good.. machine to machine DMA capability with contiguous buffers.. nothing too complicated so far.. I once did a simiar system on BSD4.3 in 1992. When I am porting my driver to FreeBSD, I use /dev/bktr driver as the example, seems easily, just using contigmalloc() to allocate the buffer, and in driver's _mmap() function, return the physical address for each page to be mmaped. The problem is, in Linux, I allocate buffer in driver's ioctl() function, and free it in a timer function, many processes may use the driver at the same time, each process use a different kernel buffer, when the process first use the driver, it calls __get_free_pages() to allocate kernel buffer, and when it exit, it trigger timer function, the timer function will can free_pages() to free the buffer, so these two kernel interfaces will be called frequently, but this usage pattern works correctly in Linux. I'm unsure about this usage of the timer (callout(?) ) how does the timer know which buffer pages to remove? and if each userspace process does an ioctl to allocate a different buffer, are the new pages also visible to other processes? in FreeBSD your driver can register with at_exit() in 4.x and with eventhandler_register() in 5.x or 6.x to have stuff done when the process exits. (there are also at_fork and at_exec handlers too.) see man at_exit (etc.) in 4.x and man 9 EVENTHANDLER in 5.x and 6.x You could also make a hash table on the PID so that you keep different information available for different processes and look them up using curproc-p_pid (for 4.x) and curthhread-td_proc-p_pid for 5.x and 6.x to look up the Using these primatives you should be able to simulate what you need I believe. In FreeBSD, the driver works in the same pattern, but when a user process mmap driver's buffer (allocated by contigmalloc()) and is killed, then when another process mmap the same buffer again, sometimes it cannot get correct data from the mmaped pages, which means the user space virtual aderess may not point to the correct physical page of driver's buffer, sometimes the OS even panic with some information such as Trap 12, Page not present etc. sounds like you are getting the buffers confused.. are you doing correc treference counting on the buffers? I browsed kernel tree, I found those drivers which use contigmalloc() and contigfree() always call these two kernel interfaces in _attach() and _detach(), but in my driver, I call contigmalloc() in ioctl(), and call contigfree() in a callout function which is set by callout_reset(). why by callout_reset()? What I want to know is if contigmalloc() and contigfree() can only be used under some conditions? not that I know of.. And recently, I modified my driver, I allocated a big chunk of contiguous physical memory using contigmalloc() in the driver's _attach() function, and I use a simple first-fit algorithm to manage this memory myself, which mean in ioctl() I use my allocate/deallocate functions instead of contigmalloc(), in _detach() function contigfree() is called to free the big chunk of memory, no panic again, but sometimes, process cannot get the correct data from the mmaped memory. I don't know why. it does sound a bit like you are not keeping the information separated between processes enough. Any help is welcomed. Thanks. _ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 10:50:26AM -0700, Julian Elischer wrote: Several times in the past we've seen people complainign that Linux allows a device driver to know who called it and somehow it seems to store somewhere some information about who openned the device.. thos somehow allows linux to store an arbitrary structure for each openning process. I thin from the sond of it that he wants to do something similar. From the sond of it he wants to have a different buffer be used depending on who is calling. This would partly work but would not work when processes fork etc. I think Linux must do some extra housekeeping in this case. What Linux does is pretty simple. The driver has access to the file structure, and this structure has a field for driver private data. It can store private data in open and free it again in the release callback. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
On Mon, Jun 13, 2005 at 05:58:24PM +0200, Hans Petter Selasky wrote:
static void
filter(struct fifo *f)
{
do {
if(!f-len)
{
if(f-m) ...;
f-m = get_mbuf();
if(!f-m) break;
f-len = m-m_len;
f-ptr = m-m_data;
}
/* f-Z_chip is the maximum transfer length */
io_len = min(f-len, f-Z_chip);
if (io_len == 0)
continue;
bus_space_write_multi_1(t,h,xxx,f-ptr,io_len);
f-len -= io_len;
f-Z_chip -= io_len;
f-ptr += io_len;
} while(Z_chip);
}
[...]
Adding that extra check for zero transfer length is not going to affect
performance at all. If one does that using C, the compiler can optimize
away that if(count) ... when count is a constant. Besides the i386
machine instructions ins and outs already exhibit that behaviour.
The compiler can only optimize it away if it is known to be a constant.
But thinking again about it, it should be documented at least whether
a count of 0 is allowed or not. I think it makes more sense to not allow
it, but others (you included) might disagree.
Joerg
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Christoph Hellwig wrote: On Mon, Jun 13, 2005 at 10:50:26AM -0700, Julian Elischer wrote: Several times in the past we've seen people complainign that Linux allows a device driver to know who called it and somehow it seems to store somewhere some information about who openned the device.. thos somehow allows linux to store an arbitrary structure for each openning process. I thin from the sond of it that he wants to do something similar. From the sond of it he wants to have a different buffer be used depending on who is calling. This would partly work but would not work when processes fork etc. I think Linux must do some extra housekeeping in this case. What Linux does is pretty simple. The driver has access to the file structure, and this structure has a field for driver private data. It can store private data in open and free it again in the release callback. How does linux handle the implications of fork(2) in this scenario? Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: prioritizing small ip packets?
IPFW does have a queue feature which is a part of dummynet. You can match packets based on size and send them to different queues. Baldur On Mon, Jun 13, 2005 at 09:36:57PM +0300, Erik Udo wrote: I came across this idea for prioritizing small IP packets, so that for example HTTP requests, game packets and other small, but importat packets would get uploaded before the big packets. Big files are usually uploaded in bigger packets, right? So, i haven't found a way to make this happen, i googled for it but didn't find anything. Does PF or IPFW have this feature? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: prioritizing small ip packets?
Baldur Gislason wrote: IPFW does have a queue feature which is a part of dummynet. You can match packets based on size and send them to different queues. Baldur On Mon, Jun 13, 2005 at 09:36:57PM +0300, Erik Udo wrote: I came across this idea for prioritizing small IP packets, so that for example HTTP requests, game packets and other small, but importat packets would get uploaded before the big packets. Big files are usually uploaded in bigger packets, right? So, i haven't found a way to make this happen, i googled for it but didn't find anything. Does PF or IPFW have this feature? I'm not sure the rationale is appropriate, though. You should be more worried about prioritizing ACKs if this is an asynchronous low-speed connection. --Devon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 12:37:07PM -0600, Scott Long wrote: How does linux handle the implications of fork(2) in this scenario? it's still counted as the same instance. Similar for dup or passing descriptors over AF_UNIX sockets. The data is explictly not per-process but per instance. There's not a lot of users actually using this feature, only the tty subsystem and multi-channel sound drivers for the old oss API that allowed multiple opens of /dev/dsp that way come to mind. Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Christoph Hellwig wrote: On Mon, Jun 13, 2005 at 12:37:07PM -0600, Scott Long wrote: How does linux handle the implications of fork(2) in this scenario? it's still counted as the same instance. Similar for dup or passing descriptors over AF_UNIX sockets. The data is explictly not per-process but per instance. There's not a lot of users actually using this feature, only the tty subsystem and multi-channel sound drivers for the old oss API that allowed multiple opens of /dev/dsp that way come to mind. Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. Ok, I thought that you were talking about per-process data being in the file descriptor. Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 12:55:40PM -0600, Scott Long wrote: Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. Ok, I thought that you were talking about per-process data being in the file descriptor. No, Linux has absolutely no concept of per-process data in driver, and if you think of it that would be rather bogus anyway (e.g. a driver opening the same device multiple times) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Christoph Hellwig wrote: On Mon, Jun 13, 2005 at 12:55:40PM -0600, Scott Long wrote: Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. Ok, I thought that you were talking about per-process data being in the file descriptor. No, Linux has absolutely no concept of per-process data in driver, and if you think of it that would be rather bogus anyway (e.g. a driver opening the same device multiple times) Hence why I was confused and asking you about it. Thanks =-) Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
On Jun 11, Dag-Erling Smrgrav wrote: Mike Hunter [EMAIL PROTECTED] writes: I have a feeling that I'm missing something really obvious, but I'm having trouble understanding why the following program: [...] Never prints anything but 0's. Because the kernel always hands processes pre-zeroed pages. I ran less up to my hw.physmem by feeding it /dev/random and watching top, and then ran the program, so I know there was tons of non-zero bits in memory. If your program had been able to see leftovers from less in its own address space, we'd have a huge security hole on our hands. I'm curious because I am worried about information leaks between processes on the same machine...did somebody decide to solve this problem while I wasn't paying attention? :) It's always been this way. Thanks for setting me straight. I guess it wasn't this way on DOS where I first learned C++ and I've assumed garbage ever since :) Is the pre-zeroing of malloc'd memory documented somewhere? By my reading of the malloc manapge... The calloc() function allocates space for number objects, each size bytes in length. The result is identical to calling malloc() with an argument of ``number * size'', with the exception that the allocated memory is explicitly initialized to zero bytes. ...it seems like it's saying that malloc (as opposed to calloc) is NOT pre-zeroed. Is there a different document I should be reading? Tussen Tak! Mike ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
Mike Hunter [EMAIL PROTECTED] wrote: On Jun 11, Dag-Erling Smrgrav wrote: Mike Hunter [EMAIL PROTECTED] writes: I have a feeling that I'm missing something really obvious, but I'm having trouble understanding why the following program: [...] Never prints anything but 0's. Because the kernel always hands processes pre-zeroed pages. Thanks for setting me straight. I guess it wasn't this way on DOS where I first learned C++ and I've assumed garbage ever since :) Is the pre-zeroing of malloc'd memory documented somewhere? By my reading of the malloc manapge... Careful now: The return value in memory from malloc() is not directly related to the return value in memory from sbrk(). malloc() may give the application back memory that was free()d previously by the same application. New pages that come out of sbrk() are 0s, but those aren't always needed to fulfill a malloc() request. The calloc() function allocates space for number objects, each size bytes in length. The result is identical to calling malloc() with an argument of ``number * size'', with the exception that the allocated memory is explicitly initialized to zero bytes. ...it seems like it's saying that malloc (as opposed to calloc) is NOT pre-zeroed. Is there a different document I should be reading? And if calloc() grabs something from the in-process used, now free pool, it will be zeroed. If malloc() grabs something from that same pool, it won't be. -- Steve Watt KD6GGD PP-ASEL-IA ICBM: 121W 56' 57.8 / 37N 20' 14.9 Internet: steve @ Watt.COM Whois: SW32 Free time? There's no such thing. It just comes in varying prices... ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
On Mon, Jun 13, 2005 at 12:31:50PM -0700, Mike Hunter wrote: On Jun 11, Dag-Erling Smrgrav wrote: Mike Hunter [EMAIL PROTECTED] writes: I have a feeling that I'm missing something really obvious, but I'm having trouble understanding why the following program: [...] Never prints anything but 0's. Because the kernel always hands processes pre-zeroed pages. I ran less up to my hw.physmem by feeding it /dev/random and watching top, and then ran the program, so I know there was tons of non-zero bits in memory. If your program had been able to see leftovers from less in its own address space, we'd have a huge security hole on our hands. I'm curious because I am worried about information leaks between processes on the same machine...did somebody decide to solve this problem while I wasn't paying attention? :) It's always been this way. Thanks for setting me straight. I guess it wasn't this way on DOS where I first learned C++ and I've assumed garbage ever since :) That is a good assumption. It is not true everywhere, but it rarely hurts being on the safe side. Is the pre-zeroing of malloc'd memory documented somewhere? By my reading of the malloc manapge... The calloc() function allocates space for number objects, each size bytes in length. The result is identical to calling malloc() with an argument of ``number * size'', with the exception that the allocated memory is explicitly initialized to zero bytes. ...it seems like it's saying that malloc (as opposed to calloc) is NOT pre-zeroed. Is there a different document I should be reading? Note that this pre-zeroing is not done by malloc, but is done by the kernel before it hands over memory to a process. Memory is not necessarily returned to the system when free() is called, but is often retained within the process and reused by the next malloc(). This means that if you have a sequence like the following: foo=malloc(1234); bar=malloc(1234); /* do something that fills the memory that foo points to with garbage */ free(foo); baz=malloc(1234); Then there is no guarantees whatsoever that baz will not point to garbage. The memory that malloc() returns in the third call to malloc() will most likely be the same as that previously pointed to by foo and will still be filled with garbage. If your program needs zeroed memory you should use calloc() or do the zeroing yourself - malloc doesn't do it. What is guaranteed is that any garbage in the memory returned by malloc() will have been created by the same process, so that information is not leaked from another process in this way. In short memory from malloc() may or may not be pre-zeroed, but it is not a security problem in either case. -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
Christoph Hellwig wrote: On Mon, Jun 13, 2005 at 12:55:40PM -0600, Scott Long wrote: Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. Ok, I thought that you were talking about per-process data being in the file descriptor. No, Linux has absolutely no concept of per-process data in driver, and if you think of it that would be rather bogus anyway (e.g. a driver opening the same device multiple times) though, some people use it for that purpose (e.g. in the original posting). it might not be such a bad idea.. I don't see why the device entrypoints shouldn't have that argument available.. (file descriptor by which we are getting here) As long as it can take account of the fact that not all accesses come via an FD (e.g mounted disks). ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 12:54:40PM -0700, Julian Elischer wrote: though, some people use it for that purpose (e.g. in the original posting). driver writers do all kinds of odd things ;-) it might not be such a bad idea.. I don't see why the device entrypoints shouldn't have that argument available.. (file descriptor by which we are getting here) As long as it can take account of the fact that not all accesses come via an FD (e.g mounted disks). disk drivers use a completely different set of entry points in Linux, and don't have access to per-fd data even in the case they're opened from userland. Character drivers to which this applies OTOH always get a valid struct file, it's guranteed as part of the driver API. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 08:59:17PM +0100, Christoph Hellwig wrote: disk drivers use a completely different set of entry points in Linux, and don't have access to per-fd data even in the case they're opened from userland. Character drivers to which this applies OTOH always get a valid struct file, it's guranteed as part of the driver API. That beeing said I'd suggest to not pass down the whole file struct if you want to add this feature for freebsd but just some well-defined API to store data in them. Giving driver writers less rope to shoot themselves improves averange driver quality significantly. We'll probably move towards such an API in Linux aswell one day, but it's a lot of work once drivers have started to do all kinds of nasty things. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: contigmalloc() and mmap()
On Mon, Jun 13, 2005 at 12:55:40PM -0600, Scott Long wrote: Lot's of driver use file-private to get at per-device data easily, but that's just a shortcut. Ok, I thought that you were talking about per-process data being in the file descriptor. Can't be done. FWIW, the main difference between FreeBSD and Linux in that area is that *all* files are vnode-based - we simply have pseudo-filesystems for pipes and sockets. So we have a single method (-release()) instead of multi-level scheme FreeBSD uses and unlike -d_close() it does see struct file * (what with being a counterpart of -fo_close()). While we are at it, is there any reason for passing struct thread * to -fo_close() and then to vop_close()? greps 1) out of -fo_close() instances only svr4_soo_close(), kqueue_close() and vn_closefile() look at the struct thread * in question. svr4_soo_close() panics if td is NULL (i.e. pass such descriptor in SCM_RIGHTS, make sure that it's garbage-collected by unp_gc() and watch closef(fp, NULL) panic the box). 2) vn_closefile() ends up passing it to VOP_CLOSE(). vop_close instances mostly ignore it or pass to other such instances. However, some do not - e.g. coda_close() panics if it gets NULL ap-a_td due to error = venus_close(vtomi(vp), cp-c_fid, flag, cred, td-td_proc); AFAICS, the only reason for passing that pointer is kludge with controlling tty handling in spec_close() (or devfs_close() in -HEAD). And it doesn't look right, even ignoring the ugliness... ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
Mike Hunter [EMAIL PROTECTED] writes: Is the pre-zeroing of malloc'd memory documented somewhere? By my reading of the malloc manapge... malloc() does not pre-zero memory, but it hands you memory which has been pre-zeroed by the kernel unless you've used it before. Your test program makes only one malloc() call, so you get memory that has never been used before. ...it seems like it's saying that malloc (as opposed to calloc) is NOT pre-zeroed. Is there a different document I should be reading? No, but nowhere in the standard does it say that memory allocated with malloc() must contain non-zero garbage. If that is what you want, though, read the TUNING section in the malloc(3) man page. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: prioritizing small ip packets?
Hi Devon, hi Baldur, (this thread would better seat on -net@, IMO) So, i haven't found a way to make this happen, i googled for it but didn't find anything. Does PF or IPFW have this feature? I'm not sure the rationale is appropriate, though. You should be more worried about prioritizing ACKs if this is an asynchronous low-speed connection. This is true, prioritizing ACKs is very useful when you want to download with full speed while uploading. But I tend to agree with Baldur's idea too : I give HTTP and DNS requests as well as interactive SSH session (TOS field set to low delay) a heavy weight in order to have them practically unaffected by a big mail delivery or a scp. Regards, -- Jeremie Le Hen jeremie at le-hen dot org ttz at chchile dot org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
On Jun 13, Erik Trulsson wrote: Is the pre-zeroing of malloc'd memory documented somewhere? By my reading of the malloc manapge... The calloc() function allocates space for number objects, each size bytes in length. The result is identical to calling malloc() with an argument of ``number * size'', with the exception that the allocated memory is explicitly initialized to zero bytes. ...it seems like it's saying that malloc (as opposed to calloc) is NOT pre-zeroed. Is there a different document I should be reading? Note that this pre-zeroing is not done by malloc, but is done by the kernel before it hands over memory to a process. Memory is not necessarily returned to the system when free() is called, but is often retained within the process and reused by the next malloc(). This means that if you have a sequence like the following: foo=malloc(1234); bar=malloc(1234); /* do something that fills the memory that foo points to with garbage */ free(foo); baz=malloc(1234); Then there is no guarantees whatsoever that baz will not point to garbage. The memory that malloc() returns in the third call to malloc() will most likely be the same as that previously pointed to by foo and will still be filled with garbage. If your program needs zeroed memory you should use calloc() or do the zeroing yourself - malloc doesn't do it. What is guaranteed is that any garbage in the memory returned by malloc() will have been created by the same process, so that information is not leaked from another process in this way. In short memory from malloc() may or may not be pre-zeroed, but it is not a security problem in either case. I got it. Thanks! This all stemmed from a discussion I was having with a coworker about vmware. I wondered aloud if information might leak from one VM to another via malloc. Whatever the answer is to that question (it's a linux VM server), I can now say I understand how FreeBSD behaves. Thanks again! Mike ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h (was: bus_at386.h)
On Monday 13 June 2005 18:27, Joerg Sonnenberger wrote:
On Mon, Jun 13, 2005 at 05:58:24PM +0200, Hans Petter Selasky wrote:
static void
filter(struct fifo *f)
{
do {
if(!f-len)
{
if(f-m) ...;
f-m = get_mbuf();
if(!f-m) break;
f-len = m-m_len;
f-ptr = m-m_data;
}
/* f-Z_chip is the maximum transfer length */
io_len = min(f-len, f-Z_chip);
if (io_len == 0)
continue;
bus_space_write_multi_1(t,h,xxx,f-ptr,io_len);
f-len -= io_len;
f-Z_chip -= io_len;
f-ptr += io_len;
} while(Z_chip);
}
[...]
Adding that extra check for zero transfer length is not going to affect
performance at all. If one does that using C, the compiler can optimize
away that if(count) ... when count is a constant. Besides the i386
machine instructions ins and outs already exhibit that behaviour.
The compiler can only optimize it away if it is known to be a constant.
But thinking again about it, it should be documented at least whether
a count of 0 is allowed or not. I think it makes more sense to not allow
it, but others (you included) might disagree.
Why?
If a count of zero is not allowed, then bus_space_xxx() should panic on
count equal to zero and not freeze the system, so that the user is able to
find out what is wrong:
#if defined(XXX)
if(count == 0) panic(not allowed);
#endif
But that is just stupid, why not just return:
#if defined(XXX)
if(count == 0) return;
#endif
And then I can put:
#define XXX
in my code before including bus.h. Instead of creating wrappers, just to be
sure:
#define bus_space_read_multi_1(t,h,off,ptr,count) \
{ if(count) bus_space_read_multi_1(t,h,off,ptr,count); }
Because this is really specific to i386 and amd64. If you look at alpha,
sparc64 and ia64 they all use while(count--) So I think the one
that wrote that code did a mistake.
My theory is that he or her copied:
static __inline void
insb(u_int port, void *addr, size_t cnt)
{
__asm __volatile(cld; rep; insb
: +D (addr), +c (cnt)
: d (port)
: memory);
}
And forgot to add that extra check for count equal to zero, when converting
rep into loop!
--HPS
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
On Tuesday 14 June 2005 01:23, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Hans Petter Selasky [EMAIL PROTECTED] writes: : So can someone have this fixed, or is there a reason not to fix it. The : one who wrote the code has done the same mistake with every one of the : bus_space_ that does memory mapped I/O. It currently breaks my : drivers. One isn't supposed to call these routines with count == 0. One could say your drivers are broken :-) Back when these were written, small optimizations like this were made to make things go faster. Now that cache sizes are bigger, a few extra instructions likely wouldn't affect things too much. Best to measure the effects of your proposed changes on real workloads... These functions are used to move smaller amounts of data. Larger amounts of data should be moved using DMA. In this case functionality is more important than performance?! -- HPS ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: prioritizing small ip packets?
On Tue, 14 Jun 2005 06:14, Jeremie Le Hen wrote: This is true, prioritizing ACKs is very useful when you want to download with full speed while uploading. But I tend to agree with Baldur's idea too : I give HTTP and DNS requests as well as interactive SSH session (TOS field set to low delay) a heavy weight in order to have them practically unaffected by a big mail delivery or a scp. There is still a problem whereby a large packet in transit can't be interrupted by a smaller packet. One solution I saw for this when doing VoIP over ADSL was using PPPoE and setting it up for multi-link over one link. The packets get fragmented into smaller pieces and the fragments of smaller packets get higher priorities. Not sure how much over head it cost, but ISTR it wasn't too bad. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C pgp9uqYAIaB9r.pgp Description: PGP signature
Re: Obvious bug in /sys/i386/include/bus.h
In message: [EMAIL PROTECTED] Hans Petter Selasky [EMAIL PROTECTED] writes: : So can someone have this fixed, or is there a reason not to fix it. The one : who wrote the code has done the same mistake with every one of the : bus_space_ that does memory mapped I/O. It currently breaks my drivers. One isn't supposed to call these routines with count == 0. One could say your drivers are broken :-) Back when these were written, small optimizations like this were made to make things go faster. Now that cache sizes are bigger, a few extra instructions likely wouldn't affect things too much. Best to measure the effects of your proposed changes on real workloads... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
In message: [EMAIL PROTECTED] Hans Petter Selasky [EMAIL PROTECTED] writes: : On Tuesday 14 June 2005 01:23, M. Warner Losh wrote: : In message: [EMAIL PROTECTED] : : Hans Petter Selasky [EMAIL PROTECTED] writes: : : So can someone have this fixed, or is there a reason not to fix it. The : : one who wrote the code has done the same mistake with every one of the : : bus_space_ that does memory mapped I/O. It currently breaks my : : drivers. : : One isn't supposed to call these routines with count == 0. One could : say your drivers are broken :-) : : Back when these were written, small optimizations like this were made : to make things go faster. Now that cache sizes are bigger, a few : extra instructions likely wouldn't affect things too much. Best to : measure the effects of your proposed changes on real workloads... : : These functions are used to move smaller amounts of data. Larger amounts of : data should be moved using DMA. In this case functionality is more important : than performance?! At the time, programmed I/O was still big, and sometimes big amounts of data were moved with them I'm not saying we can't make the interfaces more forgiving. NetBSD's implementation of these functions also have this flaw... The code looks identical, and the histories of the file leads me to believe that they were adopted unchanged from the original NetBSD implementation 7 or 8 years ago... Anyway, please view my note as a historical 'this is why it is the way it is' not a 'it must be this way because'. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
M. Warner Losh wrote: In message: [EMAIL PROTECTED] Hans Petter Selasky [EMAIL PROTECTED] writes: : So can someone have this fixed, or is there a reason not to fix it. The one : who wrote the code has done the same mistake with every one of the : bus_space_ that does memory mapped I/O. It currently breaks my drivers. One isn't supposed to call these routines with count == 0. One could say your drivers are broken :-) Back when these were written, small optimizations like this were made to make things go faster. Now that cache sizes are bigger, a few extra instructions likely wouldn't affect things too much. Best to measure the effects of your proposed changes on real workloads... Warner I'm torn between saying, this is the kernel and the kernel is an unforgiving mistress, and defensive programming is good. We still have viable and popular platforms that are based on i486, so I'd rather not see us unwind the small optimizations that are still valid there. Scott ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
In message: [EMAIL PROTECTED] Hans Petter Selasky [EMAIL PROTECTED] writes: : Where is it documented? NetBSD's man page: ... These functions will never fail. If they would fail (e.g., because of an argument error), that indicates a software bug which should cause a panic. In that case, they will never return. ... Since FreeBSD doesn't have one yet, that's as good as can be expected. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
In message: [EMAIL PROTECTED] Joerg Sonnenberger [EMAIL PROTECTED] writes: : The compiler can only optimize it away if it is known to be a constant. : But thinking again about it, it should be documented at least whether : a count of 0 is allowed or not. I think it makes more sense to not allow : it, but others (you included) might disagree. I'd be happy with commitable bus_space.9, reguardless of how this discussion turns out. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: unitialized memory is all zeros...why not garbage instead?
In message: [EMAIL PROTECTED] Mike Hunter [EMAIL PROTECTED] writes: : Is the pre-zeroing of malloc'd memory documented somewhere? By my reading : of the malloc manapge... : : The calloc() function allocates space for number objects, each size : bytes in length. The result is identical to calling malloc() with an : argument of ``number * size'', with the exception that the allocated : memory is explicitly initialized to zero bytes. : : ...it seems like it's saying that malloc (as opposed to calloc) is NOT : pre-zeroed. Is there a different document I should be reading? The memory isn't given to the process by malloc. It is given to it by some other means. That memory is zeroed for security reasons. The first time malloc returns the memory, with our current implementation, it will be all zeros. After that, all bets are off with out implementation. One should not rely on this behavior because one never knows when the first malloc happens, nor if our malloc might start writing into the memory it is about to return... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Obvious bug in /sys/i386/include/bus.h
In message: [EMAIL PROTECTED] M. Warner Losh [EMAIL PROTECTED] writes: : In message: [EMAIL PROTECTED] : Joerg Sonnenberger [EMAIL PROTECTED] writes: : : The compiler can only optimize it away if it is known to be a constant. : : But thinking again about it, it should be documented at least whether : : a count of 0 is allowed or not. I think it makes more sense to not allow : : it, but others (you included) might disagree. : : I'd be happy with commitable bus_space.9, reguardless of how this : discussion turns out. OK. Just committed one. It is silent on the issue of 0 byte counts :-) Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
