Re: [Haifux] RAM FIFO
On 4/30/05, Eli Billauer <[EMAIL PROTECTED]> wrote: > Shachar Raindel wrote: > >> On 4/27/05, Eli Billauer <[EMAIL PROTECTED]> wrote: >>> >>> >>> Hello all, >>> >>> I'm capturing a video stream with a command-line tool, which sends the >>> data stream to standard output. Now this is written to a disk, only the >>> disk access is at times too slow (the average speed is more than OK, >>> though). >>> >>> Now, since I use a common UNIX pipe to store the file (tee, actually), >>> the disk blocks, and the stream is broken. So I thought: Hmmm, how about >>> a simple application, which allocates, say, 16 MB of non-pageable RAM >>> (peanuts in today's terms), behaving like a simple data FIFO, which I >>> put in the pipeline? This would give plenty of time solve whatever is >>> more urgent than writing to the disk. >>> >>> I would write such an application, but for some reason I'm sure someone >>> has done that before me. Anyone has an idea? >> >> >> Try looking at the following: >> A. bfr, whose homepage seems to dissappear but still available from >> many distros (apt-get install bfr) >> B. mbuffer, from >> http://www.rcs.ei.tum.de/~maierkom/privat/software/mbuffer/ >> , which >> seems to do what you want >> C. buffer (I have no idea from where, just apt-get install buffer), >> which also does exactly what you want. >> >> > Thank you. > > If looks like mbuffer is the thing. Wasn't your intention to publish your > answer...? > > Oooops, should learn to click reply-all instead of replay CCing Haifux, to publish my answer... Shachar -- Haifa Linux Club Mailing List (http://www.haifux.org) To unsub send an empty message to [EMAIL PROTECTED]
Re: [Haifux] RAM FIFO
On Wednesday 27 April 2005 15:08, Eli Billauer wrote: > So I thought: Hmmm, how about a simple application, which allocates, > say, 16 MB of non-pageable RAM (peanuts in today's terms), behaving > like a simple data FIFO, which I put in the pipeline? If you have enough RAM than the non-pageable requirement isn't mandatory (it won't be paged since it's in the working set). In that case, maybe a simple dd(1) [check the ibs= and obs= options] may do the trick. -- Oron Peled Voice/Fax: +972-4-8228492 [EMAIL PROTECTED] http://www.actcom.co.il/~oron ICQ UIN: 16527398 Promises are like babies: fun to make, but hell to deliver. -- Nadav Har'El -- Haifa Linux Club Mailing List (http://www.haifux.org) To unsub send an empty message to [EMAIL PROTECTED]
Re: [Haifux] RAM FIFO
On Wed, Apr 27, 2005 at 02:08:55PM +0200, Eli Billauer wrote: > Now, since I use a common UNIX pipe to store the file (tee, actually), > the disk blocks, and the stream is broken. So I thought: Hmmm, how > about a simple application, which allocates, say, 16 MB of > non-pageable RAM (peanuts in today's terms), behaving like a simple > data FIFO, which I put in the pipeline? This would give plenty of time > solve whatever is more urgent than writing to the disk. I am not familiar with such an application but it should be trivial to write. Two other things you might want to consider: - tee sucks for performance. Unless you have particular reason to use tee, I recommend doing stdio -> file, and tail -f on the file seperately. - 2.6.11 and onwards kernels have a much improved pipe handling code. If the disk isn't your only bottleneck, you might want to try this. Also, you can play various games with the disk and application in question to reduce (or eliminate) the disk problems. Things you can try include hdparm, elevator settings, AIO and DIO. Cheers, Muli -- Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/ -- Haifa Linux Club Mailing List (http://www.haifux.org) To unsub send an empty message to [EMAIL PROTECTED]
[Haifux] RAM FIFO
Hello all, I'm capturing a video stream with a command-line tool, which sends the data stream to standard output. Now this is written to a disk, only the disk access is at times too slow (the average speed is more than OK, though). Now, since I use a common UNIX pipe to store the file (tee, actually), the disk blocks, and the stream is broken. So I thought: Hmmm, how about a simple application, which allocates, say, 16 MB of non-pageable RAM (peanuts in today's terms), behaving like a simple data FIFO, which I put in the pipeline? This would give plenty of time solve whatever is more urgent than writing to the disk. I would write such an application, but for some reason I'm sure someone has done that before me. Anyone has an idea? BTW -- my disk is being defragmented as these lines are written. Thanks, Eli -- Web: http://www.billauer.co.il -- Haifa Linux Club Mailing List (http://www.haifux.org) To unsub send an empty message to [EMAIL PROTECTED]
