Multithreaded server performance
I am currently porting a multithreaded TCP server from NT (yech!) to UNIX using pthreads. The server has a fairly straightforward design -- it opens a thread for each connection, and each thread spends most of its life blocked in a call to read() from a socket. As soon as it receives enough of a request, it does quite a bit of processing and sends something back to the client. How would FreeBSD 4.0 perform in such a scenario? We are talking hundreds, maybe thousands of threads, a lot of them doing blocking reads and writes. Is the standard pthreads library adequate for the task, or would "Linuxthreads" be a better choice? What is the main difference between the standard FreeBSD pthreads and "Linuxthreads" -- it seems both are implemented using a version of clone(). The hardware is probably going to be UP at first -- a fast Pentium III (733MHz?), an Intel 820 board, an Adaptec U2W SCSI controller, and a couple of U2W LVD SCA disks to go with it. The operating system has yet to be chosen. I have tried Solaris 7 on similar hardware, and it seems so much slower than FreeBSD -- and so bloody unresponsive when doing I/O that even NT seems faster. On the other hand, Solaris's threads implementation is supposedly better than anything else out there. I'm not even considering Linux -- or should I? Is it safe to use FreeBSD 4.0-S on a production server? So far I've had nothing but positive experience with it. Any advice will be much appreciated. Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Multithreaded server performance
Jason and Richard, thank you very much for your explanations of libc_r and LinuxThreads. Due to the significant processing time of each request (typically between 50-800ms, averaging 100ms), I doubt that FreeBSD's threads would perform any worse than if I forked a separate process for each connection (memory usage would go sky high), or even if I had a single process and called poll() myself to check each descriptor for being readable/writable. What worries me though is that by design client connections are kept alive (although the server is allowed to disconnect a client after a period of inactivity), hence after a while there will be a lot of idle descriptors, and continuously polling them might slow down the processing of the active ones. I have to experiment to find out what's best -- the goal is to do better than NT (which surprisingly does quite a good job when it comes to processing lots of threads). Solaris's threads are pretty darn good too, but I dislike all things SystemV-ish, and Solaris/x86 isn't all that hot either (compared with the version for the UltraSPARC). > On Sun, Apr 23, 2000 at 09:21:15PM -0700, Jason Evans wrote: > > On Mon, Apr 24, 2000 at 05:17:10AM +0300, A G F Keahan wrote: > > > I am currently porting a multithreaded TCP server from NT (yech!) to > > > UNIX using pthreads. The server has a fairly straightforward design -- > > > it opens a thread for each connection, and each thread spends most of > > > its life blocked in a call to read() from a socket. As soon as it > > > receives enough of a request, it does quite a bit of processing and > > > sends something back to the client. > > > > This design isn't ideal on any OS, but the fact that you do significant > > processing every time a request arrives on a socket probably hides most of > > the inefficiency due to thread switching and lack of cache locality due to > > many thread stacks. Can you suggest a better design for this type of server application under FreeBSD?Perhaps a combination of forking (or pre-forking) and threads? > > > How would FreeBSD 4.0 perform in such a scenario? We are talking > > > hundreds, maybe thousands of threads, a lot of them doing blocking reads > > > and writes. Is the standard pthreads library adequate for the task, or > > > would "Linuxthreads" be a better choice? What is the main difference > > > between the standard FreeBSD pthreads and "Linuxthreads" -- it seems > > > both are implemented using a version of clone(). > > > > FreeBSD's threads should perform adequately given the design of your program > > and the hardware you listed. Actually trying it on the various operating > > systems would be a good exercise, but I have found that FreeBSD's threads > > perform at least as well as Solaris's for such an application. > > Interesting. I would have thought Solaris would be much superior. I would have thought so too... we'll see. Thanks again Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: new device drivers for RME soundcard and G400-TV
Daniel,
First of all, I think you will get a lot more replies if you re-post
your message to freebsd-multimedia (cc'd). Secondly, the author and
maintainer of the new pcm driver in 4-stable and -current is Cameron
Grant ([EMAIL PROTECTED]), and you should talk to him before starting
anything serious. The sound driver in FreeBSD 3.4 (and earlier) is
unmaintained and basically dead -- if your system is 3-stable or older,
go -current, then backport your changes to 4-stable.
I believe that newpcm was specifically designed to support advanced
features of modern sound cards, and also to be easily extensible. The
only functionality currently missing is MIDI, and you should talk to
Seigo Tanimura about it (he has some patches to fix it). I also
believe (and I'm sure that many people will agree with me) that we
should try to stay clear of GPLd code unless absolutely unavoidable.
FreeBSD's driver framework is sufficiently different from that of Linux
that in practice it's not much of a problem. On the other hand, if
porting the ALSA library would buy us support for a whole lot of new
soundcards, it might be well worth it.
Personally, I think that your time would be better spent improving the
existing system rather than designing a new one. If you discover that
newpcm is not powerful enough to accommodate some of the more advanced
features of your card, then we should find a way to improve it.
Good luck
Alex Keahan
Daniel Pouzzner wrote:
>
> I'm about to get cracking on a device driver for the RME Digi96/8 PAD.
>
> The card's full set of I/O channels is: S/PDIF I/O (2 channels in, 2
> out, 16-24 bits@32-96kHz), AES/EBU I/O (2ch in, 2 out, 16-24
> bits@32-96kHz), analog I/O (2ch in, 2 out, 16-24 bits@32-96kHz), and
> ADAT I/O (8 channels in, 8 out, 16-24 [EMAIL PROTECTED] or 48kHz).
>
> The hardware supports on-board routing between these channels,
> e.g. ADAT->S/PDIF or S/PDIF->ADAT. I will be supporting these
> features. For those who are simply curious about features, see
> http://www.rme-audio.com/english/digi96/digi96pa.htm
>
> My starting point (what I have in hand) is (1) a complete and
> functioning driver (supporting most key functionalities of the card)
> for Linux and the ALSA ("Advanced Linux Sound Architecture")
> subsystem, and (2) complete internal documentation on the card, from
> the manufacturer.
>
> Is FreeBSD's OSS-like audio subsystem powerful enough to provide
> access to all the card's features (particularly, all its ports, word
> sizes, and sample rates)? ac97.c goes only to 20 bits/sample, for
> example, but the card is 24 bits throughout.
>
> I have to decide what strategy to pursue: do I write a new driver
> based on an existing FreeBSD driver, using the Linux driver for hints,
> or do I use a FreeBSD driver for hints, basing the new driver on the
> Linux one (and thereby winding up with a GPL'd driver)? Or indeed do
> I write a driver truly from the ground up? The only FreeBSD soundcard
> driver I know of that includes analog and digital I/O support is the
> SB Live! driver, which in my experience doesn't work particularly well
> (in particular, capture didn't work at all when I tried it in April),
> so my sense is that there exists no smoothly working FreeBSD driver
> for a soundcard with anything remotely approaching the capabilities of
> the RME products.
>
> Finally, I have to decide whether I will port the ALSA library (in
> whole or in part - appealing because it would facilitate support for
> the dozens of other soundcards ALSA supports and FreeBSD doesn't),
> write a glue layer so that a driver that supplies an ALSA-type
> interface will work with the FreeBSD audio subsystem, simply create a
> driver that supplies a FreeBSD-style audio interface, or finally,
> create a driver that allows direct and specialized ioctl access to
> unusual card features not accommodated by the FreeBSD audio subsystem,
> but otherwise providing access via /dev/dsp et al.
>
> This will be the first device driver I write. Any advice, assistance,
> caveats, etc., will certainly be much appreciated. In particular, if
> anyone has a guide to device driver implementation, draft or polished
> form, complete or incomplete, it would surely be helpful to me. I
> understand some basic porting issues, e.g. bus_space_read_4 instead of
> readl(), copyin() instead of copy_from_user(), etc. The details will
> come into focus with determination and repeated exposure.
>
> I already have the card on the PCI bus, and am quite committed to BSD,
> so I will see this project to its successful conclusion (unless
> someone beats me to it (by all means, go ahead!:-)).
>
> -Daniel Pouzzner
>
> P.S. Also, if no one beats me to it, I will port G400-TV support for
> the realtime MJPEG codec, the TV/radio tuner, and video I/O and audio
> routing. The Linux development effort
> (http://marvel.sourceforge.net/) is just coming together and I won't
> be embarking on the project until they've got t
Re: Linux emu question
James, You can download a fully functional 30-day evaluation version of VMWare and see for yourself (make sure you tell them that your "distribution of Linux" is FreeBSD). There are two ports in the FreeBSD ports collection -- /usr/ports/emulators/vmware1 and vmware2, which make installation a lot easier. I haven't tried VMWare2, because VMWare1 does everything I need and I don't want to fix what ain't broken. NT4.0 works very well in a virtual machine (with VMWare tools installed), just a bit slower than running on the actual hardware. Make sure your graphics adapter supports DGA (I have Matrox G200), and your disks are fast (mine are Ultra66 DMA). You also need to mount linprocfs, and install the rtc device (also in ports). Alex P.S. BTW, questions about Linux emulation should be sent to freebsd-emulation. James Halstead wrote: > > I noticed some people talking about the linux emulation and how good/bad it > can be and I just wondered, does anybody here have any experiences with the > vmware for linux software? I have been thinking of buying this, for those > one or two windows programs that I need to use now and then. > > Just wondering, > James. > --- > ICQ #19675056 > Public key available at: > http://www.dreamscape.com/halstead/jh.asc > --- To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Generic config file parser?
This may be a silly question, but is there such a thing?Almost every program that I know uses configuration files, often in different, incompatible formats. I personally prefer Samba/Wine-style config files which are split into "sections" like this: [SECTIONNAME1] wibble1 = blah wibble2 = 35 wibble3 = "a string that has more than one word" ; this is a comment -- ignored wibble4 = 4.567e9 What I'm after is some kind of a generic parser function, which you can give a section name "SECTIONNAME1"), a token name (e.g. "wibble1"), a separator (e.g. "=", ":", or white space ""), and a variable of some type (char *, int, float, etc), and have it return "true" and the requested value if it exists, or "false" if the value was not found or datatype was incorrectly specified. Similarly, I want to be able to save a value into a given section of the file, possibly replacing an existing value, or adding a new one. A trivial thing to write, and a very useful one -- surely it's been done before, no? Alex To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Generic config file parser?
Doug Barton wrote: > > A G F Keahan wrote: > > > > This may be a silly question, but is there such a thing?Almost every > > program that I know uses configuration files, often in different, > > incompatible formats. I personally prefer Samba/Wine-style config > > files which are split into "sections" like this: > > > > [SECTIONNAME1] > > wibble1 = blah > > wibble2 = 35 > > wibble3 = "a string that has more than one word" > > ; this is a comment -- ignored > > wibble4 = 4.567e9 > > > A trivial thing to write, and a very useful one -- surely it's been done > > before, no? > > It's possible, though unlikely, since what you're describing is a > windows style config format. Which does not necessarily make it bad, of course, it's just nice to separate things out. Many UNIX applications use similar formats; you don't have to go very far to find an example -- look at e.g. /etc/defaults/rc.conf, which has distinct sections separated by comments like this one: ## ### Important initial Boot-time options ## I'll say it again -- I'm not specifically after a Windows-style config file parser, just any similar type thing. Alex To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Optimal UFS parameters
What parameters should I choose for a large (say, 60 or 80Gb) filesystem? I remember a while ago someone (phk?) conducted a survey, but nothing seems to have come of it. In the meantime, the capacity of an average hard drive has increased tenfold, and the defaults have become even less reasonable. What's the current consensus of opinion? newfs -b ? -f ? -c ? Thanks Alex Keahan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
A question about multithreaded server
I have a server built around the following model:
One "manager" thread selecting on multiple descriptors
(using kqueue() on FreeBSD, poll() on Solaris,
select() on some other Unix systems, and
WSAWaitForMultipleEvents() on Win32).
A small number (typically 2 x the number of CPUs) of
worker threads doing the actual work.
The manager thread handles accepting new connections
(the listening descriptor is also included in the
select loop), as well as reading and sending. The
read handler knows when a complete request has arrived
and dispatches it to one of the worker threads
(sleeping on a condvar), waking it up. The file
descriptors and their states (e.g. read and send
buffers) are stored in an array. If all workers are
busy, the connection's index is placed in a global run
queue (which along with its counter is protected by a
mutex). When a worker finishes processing a
connection and has nothing else to do, it is allowed
to get an item from the run queue.
The connection array has a fixed size, and connections
are not protected by any mutices, with the following
restriction: the selecting thread ("manager") is not
allowed to touch the read and send buffers of a
connection that is currently being processed by a
worker (so reading and writing is serialized: read -
work - write - read - ...)
The problem I'm having with this model is that I don't
see an efficient way of telling the manager that a
worker has finished processing a connection and that
the data is ready to be sent back to the client.
When a connection is dispatched to a worker, it is
deleted from the select queue, because we don't care
if it's ready for reading or writing until the worker
is done processing the data (and even if we cared, we
are still not allowed to read() or write() to the
connection buffers). Unfortunately, the worker thread
cannot add the connection back to the select queue,
because select() will break if the data is changed
from under it. But how can I tell select() "wake up,
the data is here" from another thread without breaking
it?
One solution is not to delete the worker connections
from the queue, but give select() a reasonably small
timeout and poll the workers for data every so and so
milliseconds. If a connection is ready for writing
but still has no data to send, ignore it and go around
the loop once again. Unfortunately, such polls steal
CPU time from the workers and lead to noticeable
delays.
Perhaps a better solution would be to add another file
descriptor (let's call it notify_fd) to the select
queue, and have the workers write to it on finishing
the processing, immediately waking up the manager
thread blocked on select. When the manager wakes up,
it checks if it was woken up by notify_fd, and if so,
goes through all the workers, adding their fds back to
the select queue if their data status is marked
"ready". Then it processes other active connections
and once again goes around the loop, but this time, if
any of the worker connections are ready for writing,
they will actually have some data to write.
The question is -- what can I use as a dummy
"notify_fd" descriptor? Another socket? A Unix
domain socket? Does this sound like a good solution?
Is there a better way to achieve the same result?
Any suggestions will be greatly appreciated.
Alex Keahan
Nokia Game is on again.
Go to http://uk.yahoo.com/nokiagame/ and join the new
all media adventure before November 3rd.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
