Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
Quoth Unga unga...@yahoo.com: I think you may be reading too much into the malloc manpage.� When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety.� Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. Good to note all allocations are thread safe in FreeBSD. Is it by some standard that malloc should be thread safe regardless the OS (BSDs, Linux, Windows, Android, etc)? POSIX (well, SUSv4 at least) says that malloc and free must be threadsafe. Note that Windows is not a POSIX system, though I belive malloc is also always threadsafe on Windows. Ben ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: FreeBSD 9.1 excessive memory allocations [SOLVED]
On Wed, Mar 27, 2013 at 10:39:16PM -0700, Unga wrote: I think you may be reading too much into the malloc manpage.? When it mentions the use of per-thread small-object caches to avoid locking it's talking about performance, not thread safety.? Allocations of all sizes are thread-safe, the library just assumes that huge allocations are rare enough that it doesn't use extra per-thread resources to avoid locking for them, it just uses locking for huge blocks. -- Ian Good to note all allocations are thread safe in FreeBSD. Is it by some standard that malloc should be thread safe regardless the OS (BSDs, Linux, Windows, Android, etc)? The Internet is sometimes a useful resource. http://stackoverflow.com/questions/855763/malloc-thread-safe http://stackoverflow.com/questions/987444/about-thread-safety-in-malloc-and-free http://msdn.microsoft.com/en-us/library/ms235505.aspx http://msdn.microsoft.com/en-us/library/abx4dbyh.aspx On the BSDs, with gcc -pthread you should be fine. Don't ask me about Clang/LLVM. On Linux, glibc's routines are apparently thread-safe barring if the software was built with -DNO_THREADS (which seems very unlikely given the risks). Android is Linux. On Windows, assuming you're using CRT libraries, the routines are thread-safe. I get the impression the threaded library versions are also thread-safe for their *alloc equivalents. I believe HeapAlloc is thread-safe, and am pretty sure Global* is thread-safe (barring very very old Wine environments). Solaris's routines are also thread-safe, barring you're using standard malloc and not mtmalloc or ptmalloc or libumem (try searching the web for things like Solaris mtmalloc and read). All these OSes in some way or another have support for multi-threaded malloc implementations for increased performance (this is separate from locking/lock contention) where, AFAIK, you get to do the locking yourself. -- | Jeremy Chadwick j...@koitsu.org | | UNIX Systems Administratorhttp://jdc.koitsu.org/ | | Mountain View, CA, US| | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On 28.03.2013 02:43, Adrian Chadd wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. Are there many boards now with ATA, but without USB? But I agree, it should be checked. -- Alexander Motin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
Alexander Motin wrote this message on Thu, Mar 28, 2013 at 09:17 +0200: On 28.03.2013 02:43, Adrian Chadd wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. Are there many boards now with ATA, but without USB? But I agree, it should be checked. The net4501 board has ATA but no USB.. Also, depending upon use, you might choose to not include USB, but use ATA, or not use umass, but the rest of USB... Someone on a list was talking about trying to get FreeBSD down on a really small system, 16MB ram... /me thinks of the old wd driver. -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: SU+J hard recovery failure
Le jeudi 28 mars 2013 11:05:58 Daniel O'Connor a écrit : On 27/03/2013, at 18:43, David Demelier demelier.da...@gmail.com wrote: Yesterday I had a panic on my laptop. Unfortunately the SU+J was not able to recovery the file system, the error was something like : Unknown error: Help! Could not find directory 9854215 And it went to the single user mode, I needed to fsck_ufs manually to recover the disk. What afraid me is what if happens on a headless machine? It will be hard to recover. For headless remote systems I use fsck_y_enable=YES. In my experience 'fsck -y' almost invariably recovers the disk to a point where it can boot and then I can login remotely. When the problem appeared, I could not use fsck -y because it will asks Use journal? and if journal is used the error above reappears and you can not clean the file system.. I needed to do a full fsck without any journal at all. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C Regards, -- David Demelier ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On Thu, 2013-03-28 at 09:17 +0200, Alexander Motin wrote: On 28.03.2013 02:43, Adrian Chadd wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. Are there many boards now with ATA, but without USB? But I agree, it should be checked. It's not necessarily what the boards have but how they're used. We use industrial SBCs at work that have ata compact flash sockets on the board which we do use, and usb interfaces which we don't use. I've never tested the new ata+cam stuff on some of these boards, most based on Cyrix, Via, Geode, and VortexD86 chipsets. The older ata code works, but not always very well -- for example, we usually have to set hw.ata.ata_dma=0 for absolutely no reason we've ever been able to figure out except that if we leave it enabled we get DMA errors and panics on some CF cards and not on others. I have no idea whether to expect such things to be better, worse, or no different by changing to the ata+cam way of doing things (but I don't really have time to do extensive testing right now either). -- Ian ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On Wed, 27 Mar 2013 17:43:07 -0700 Adrian Chadd adr...@freebsd.org wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. /me never seen embedded devices with ATA/SATA and less than 64MB of RAM. (i386/i486 old machines does not count :) ) I'm missing something? Thanks, adrian ___ freebsd-curr...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Aleksandr Rybalko r...@ddteam.net ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On Mar 27, 2013, at 6:43 PM, Adrian Chadd adr...@freebsd.org wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. From a code execution standpoint? No, it's not. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. From a code segment size standpoint, there's definitely some stuff that should be made modular and optional. Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On Mar 28, 2013, at 8:00 AM, Ian Lepore i...@freebsd.org wrote: On Thu, 2013-03-28 at 09:17 +0200, Alexander Motin wrote: On 28.03.2013 02:43, Adrian Chadd wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. Are there many boards now with ATA, but without USB? But I agree, it should be checked. It's not necessarily what the boards have but how they're used. We use industrial SBCs at work that have ata compact flash sockets on the board which we do use, and usb interfaces which we don't use. I've never tested the new ata+cam stuff on some of these boards, most based on Cyrix, Via, Geode, and VortexD86 chipsets. The older ata code works, but not always very well -- for example, we usually have to set hw.ata.ata_dma=0 for absolutely no reason we've ever been able to figure out except that if we leave it enabled we get DMA errors and panics on some CF cards and not on others. I have no idea whether to expect such things to be better, worse, or no different by changing to the ata+cam way of doing things (but I don't really have time to do extensive testing right now either). The legacy ATA code was hard to maintain, very buggy (as you point out), and is essentially unmaintained. Also, IIRC, the legacy stack simply cannot support NCQ tagged queueing. I think that Alexander has done a superb job with both developing and supporting the CAM_ATA stack. Scott ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
Hello, Aleksandr. You wrote 28 марта 2013 г., 18:09:53: It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. AR /me never seen embedded devices with ATA/SATA and less than 64MB of RAM. AR (i386/i486 old machines does not count :) ) AR I'm missing something? Yes: USB UMASS. It uses CAM too, and useful for very small systems, like 4MiB FLASH and 16MiB RAM (yes, whole system image, kernel and all, should be packed to 4MiB). Please note, Adrian speaks about CAM, not only CAM + ATA. -- // Black Lion AKA Lev Serebryakov l...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On 28 March 2013 09:05, Lev Serebryakov l...@freebsd.org wrote:
Yes: USB UMASS. It uses CAM too, and useful for very small systems,
like 4MiB FLASH and 16MiB RAM (yes, whole system image, kernel and
all, should be packed to 4MiB).
Please note, Adrian speaks about CAM, not only CAM + ATA.
And I'm not at all saying we should keep the old ATA driver around.
I'm just pointing out a set of use cases that most FreeBSD developers
aren't involved with and I'd like to find a way to squeeze it more
efficiently into embedded platforms.
I've never had any noticable performance issues with CAM on my
embedded MIPS boards because it's typically pushing packets. It's just
the resultant binary size of the whole stack that's a problem.
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | grep scsi
textdata bss dec hex filename
49372 10672 80 60124eadc scsi_all.o
212002576 16 237925cf0 scsi_da.o
232881488 16 2479260d8 scsi_xpt.o
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | grep cam
textdata bss dec hex filename
3824 96 163936 f60 cam.o
13552 144 16 137123590 cam_periph.o
2344 144 02488 9b8 cam_queue.o
640 48 0 688 2b0 cam_sim.o
40684 752 192 41628a29c cam_xpt.o
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | grep umass
textdata bss dec hex filename
225921072 16 236805c80 umass.o
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | egrep
'(cam_|umass|scsi_)'
13552 144 16 137123590 cam_periph.o
2344 144 02488 9b8 cam_queue.o
640 48 0 688 2b0 cam_sim.o
40684 752 192 41628a29c cam_xpt.o
49372 10672 80 60124eadc scsi_all.o
212002576 16 237925cf0 scsi_da.o
232881488 16 2479260d8 scsi_xpt.o
225921072 16 236805c80 umass.o
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | egrep
'(cam_|umass|scsi_)' | awk '{a+=$4} END {print a}'
190904
It doesn't seem like a lot, but it does add up..
Adrian
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
.. and before you ask - yes, there are embedded boards with limited RAM that also have ATA ports. :-) Adrian ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
In message
CAJ-Vmo=qATZHubkKZ2heiJ3528e__JG4RLru7LU9rwP5_EwT=g...@mail.gmail.com, Adrian
Chadd wri
tes:
On 28 March 2013 09:05, Lev Serebryakov l...@freebsd.org wrote:
adrian@freefall:~/public_html/ath$ cat AP121-nodebug.txt | egrep
'(cam_|umass|scsi_)' | awk '{a+=$4} END {print a}'
190904
It doesn't seem like a lot, but it does add up..
Isn't there some kernel compile-time option to eliminate the huge
tables used for errormessages etc ?
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On Thu, 28 Mar 2013, Ian Lepore wrote: On Thu, 2013-03-28 at 09:17 +0200, Alexander Motin wrote: On 28.03.2013 02:43, Adrian Chadd wrote: My main concern with the new stuff is that it requires CAM and that's reasonably big compared to the standalone ATA code. It'd be nice if we could slim down the CAM stack a bit first; it makes embedding it on the smaller devices really freaking painful. Are there many boards now with ATA, but without USB? But I agree, it should be checked. It's not necessarily what the boards have but how they're used. We use industrial SBCs at work that have ata compact flash sockets on the board which we do use, and usb interfaces which we don't use. I've never tested the new ata+cam stuff on some of these boards, most based on Cyrix, Via, Geode, and VortexD86 chipsets. The older ata code works, but not always very well -- for example, we usually have to set hw.ata.ata_dma=0 for absolutely no reason we've ever been able to figure out except that if we leave it enabled we get DMA errors and panics on some CF cards and not on others. I have no idea whether to expect such things to be better, worse, or no different by changing to the ata+cam way of doing things (but I don't really have time to do extensive testing right now either). Woa, I have to set hw.ata.ata_dma=0 also in order to get FreeBSD to boot on a PC104 board. I think ours is a Cyrix or Via also. -- DE ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Any objections/comments on axing out old ATA stack?
On 28 March 2013 10:26, Poul-Henning Kamp p...@phk.freebsd.dk wrote: Isn't there some kernel compile-time option to eliminate the huge tables used for errormessages etc ? Yup. It doesn't save all that much in the grand scheme of things. Doubly so since my secondary size constraint is an 896k partition that I lzma compress the kernel to fit into. Those strings don't add much to the final lzma image size. Adrian ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
