Re: [LAD] [OT] bash vs disk activity
On Thu, 2007-10-11 at 13:42 -0400, Ross Vandegrift wrote:
> On Tue, Oct 09, 2007 at 06:58:40PM +0700, Patrick Shirkey wrote:
> > Does anyone have a script for monitoring with iostat or vmstat while in
> > async mode?
> >
> > I'm thinking a bash equivalent to the disk monitor on gkrellm which
> > allows me to know when a transfer has completed.
>
> Something like this will return when five seconds have passed with
> no write requests being issues to the device sda:
>
> CURRENT=$(cat /proc/diskstats | grep sda\ | awk '{ print $8 }')
> LAST=0
> while true; do
> LAST=CURRENT
> sleep 5
> CURRENT=$(cat /proc/diskstats | grep sda\ | awk '{ print $8 }')
> if [ "$LAST" == "$CURRENT" ]; then
> break
> fi
> done
>
>
> /proc/diskstats lists a few things. For complete devices (ie,
> /dev/sda instead of /dev/sda1), it lists:
>
> Field 1 -- # of reads issued
> Field 2 -- # of reads merged, field 6 -- # of writes merged
> Field 3 -- # of sectors read
> Field 4 -- # of milliseconds spent reading
> Field 5 -- # of writes completed
> Field 7 -- # of sectors written
> Field 8 -- # of milliseconds spent writing
> Field 9 -- # of I/Os currently in progress
> Field 10 -- # of milliseconds spent doing I/Os
> Field 11 -- weighted # of milliseconds spent doing I/Os
>
>
> Where field n is the nth field after the device name.
>
Hi,
Thanks everyone again for your suggestions. The above is exactly what I
was thinking of. Thanks very much Ross. I owe you a beer.
I had to make a couple of small changes.
CURRENT=$(cat /proc/diskstats | grep sdb\ | awk '{ print $8 }')
LAST=0
while true; do
echo -en l = $LAST c = $CURRENT \\n
sleep 5
CURRENT=$(cat /proc/diskstats | grep sdb\ | awk '{ print $8 }')
if [ $LAST == $CURRENT ]; then
break
fi
LAST=$CURRENT
done
Cheers.
--
Patrick Shirkey
Boost Hardware Ltd.
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
On Tue, Oct 09, 2007 at 06:58:40PM +0700, Patrick Shirkey wrote:
> Does anyone have a script for monitoring with iostat or vmstat while in
> async mode?
>
> I'm thinking a bash equivalent to the disk monitor on gkrellm which
> allows me to know when a transfer has completed.
Something like this will return when five seconds have passed with
no write requests being issues to the device sda:
CURRENT=$(cat /proc/diskstats | grep sda\ | awk '{ print $8 }')
LAST=0
while true; do
LAST=CURRENT
sleep 5
CURRENT=$(cat /proc/diskstats | grep sda\ | awk '{ print $8 }')
if [ "$LAST" == "$CURRENT" ]; then
break
fi
done
/proc/diskstats lists a few things. For complete devices (ie,
/dev/sda instead of /dev/sda1), it lists:
Field 1 -- # of reads issued
Field 2 -- # of reads merged, field 6 -- # of writes merged
Field 3 -- # of sectors read
Field 4 -- # of milliseconds spent reading
Field 5 -- # of writes completed
Field 7 -- # of sectors written
Field 8 -- # of milliseconds spent writing
Field 9 -- # of I/Os currently in progress
Field 10 -- # of milliseconds spent doing I/Os
Field 11 -- weighted # of milliseconds spent doing I/Os
Where field n is the nth field after the device name.
--
Ross Vandegrift
[EMAIL PROTECTED]
"The good Christian should beware of mathematicians, and all those who
make empty prophecies. The danger already exists that the mathematicians
have made a covenant with the devil to darken the spirit and to confine
man in the bonds of Hell."
--St. Augustine, De Genesi ad Litteram, Book II, xviii, 37
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] VroomBox
Does it at all need impulse response convolver as suggested...? How about just tracking the engine pitch and having a sampler to do the sound? LinuxSampler for fine details. The device may be illegal in Europe. Juhana -- http://music.columbia.edu/mailman/listinfo/linux-graphics-dev for developers of open source graphics software ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
[LAD] maintenance @ linuxaudio.org
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hooray, To provide better performance and reliability linuxaudio.org is replacing old hardware and upgrading the network bandwidth ! Scheduled server maintenance - Fri Oct 12 2007. The servers will be down for a short time around 18.00 CEST, 12pm EST or 9am PST. The HTTP services will resume almost immediately (reboot). but the mailing lists might be off-line for up to 3 hours. During this time the list server will not accept incoming messages and will not distribute messages. Any messages sent to the lists should be queued by the sending mail servers and will be accepted once the new list server is brought into service. #robin, for the linuxaudio.org team -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHDjxieVUk8U+VK0IRAobPAJ9OgEje2zELBE5Bug1Y3f6gTJgD1QCfQx+I Q/5QoxJoq63BEcg4rP64uzY= =5saU -END PGP SIGNATURE- ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
Hello Patrick, How would I track the 5 minutes of data transfer with bash? The only reliable way on unix to ensure all data has been written to the disk is to umount it. The "sync" (API call or program) is merely a hint to flush filesystem buffers at the point where the call was made, but one second later, while the sync is still running, new dirty buffers might be generated. So you should use something along the following: time ( mount /dev/xxx /mnt/yyy ; cp -a /whatever /mnt/yyy ; umount /mnt/yyy ) -- ---> Dirk Jagdmann > http://cubic.org/~doj -> http://llg.cubic.org ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
Hi, On Thu, Oct 11, 2007 at 04:43:23AM -0400, Paul Davis wrote: > On Thu, 2007-10-11 at 11:52 +0700, Patrick Shirkey wrote: > > On Wed, 2007-10-10 at 06:37 -0400, Forest Bond wrote: > > > Hi, > > > > > > On Wed, Oct 10, 2007 at 09:51:35AM +0700, Patrick Shirkey wrote: > > > > On Tue, 2007-10-09 at 09:16 -0400, Forest Bond wrote: > > > > > On Tue, Oct 09, 2007 at 01:11:38PM +0200, Lars Luthman wrote: > > > > > > Slightly-slower-hack: add the 'sync' option to whatever script or > > > > > > config > > > > > > file is used to mount the device (/etc/fstab, udev rules, KDE > > > > > > service > > > > > > menu etc). That way all writes will be synchronised automatically. > > > > > > > > > > It sounds like this doesn't matter much for you, but keep in mind > > > > > that -o > > > > > sync doesn't cause reads to be unbuffered, so don't go trying to > > > > > benchmark > > > > > your flash devices that way! > > > > > > > > Hmm, I need speed and accuracy. > > > > > > But only for writes, no? > > > > For the purpose of this script I am happy with just write but I will be > > needing read in the near future. > > > > FYI, I can connect 200 usb-storage devices to a dual cpu intel with 4 GB > > RAM and an extra pci-usb card on the same irq as the internal usb bus. > > > > Basically I am uploading content to a couple of hundred mp3/multi media > > players at the same time and this script is used by other people with > > less Linux knowledge than I so I'm trying to make the progress bar as > > accurate as possible. Otherwise there is a significant lag between when > > the progress bar says it is finished to when the transfer is actually > > finished. Speed is also a major issue here so I cannot use the sync > > option as that causes way too much delay compared to async. > > > > Can anyone here shed a little more light on how to monitor the disk > > activity while transferring data? > > if you use your own compiled code for the copy, you can open the > destination files with O_SYNC. from man 2 open: > > O_SYNC The file is opened for synchronous I/O. Any write()s on > the resulting file descriptor will block the calling process > until the data has been physically written to the underlying > hardware. See RESTRICTIONS below, though. > > this avoids having the entire mount be done with the sync option. Or, use Python instead of bash... Python gets you a full POSIX interface, and you can work a shell from within Python (when you need it). Alternatively, you could whip up a very small Python/C/whatever program that you could call from your shell script. It could even print the percentages to stdout or stderr, making your shell script's life easier: $ copy-sync source dest 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
On Thu, 2007-10-11 at 10:32 +0200, Luis Garrido wrote: > > Can anyone here shed a little more light on how to monitor the disk > > activity while transferring data? > > > > Not sure if this would work, but first thing I can think of is: > > 1) Calculate the amount of data to transfer (du command.) > 2) Find out the remaining free space in the target unit (df command.) > 3) Calculate the remaining free space when the copy is done. > 4) Begin copy. > 5) Periodically monitor the free space. Hi Luis, Thanks for your suggestion. I have tried that already but the cp command grabs the space allocated while it is buffering the data to transfer so as far as df is concerned the space has been filled :/ Cheers. -- Patrick Shirkey Boost Hardware Ltd. ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
On Thu, 2007-10-11 at 11:52 +0700, Patrick Shirkey wrote: > On Wed, 2007-10-10 at 06:37 -0400, Forest Bond wrote: > > Hi, > > > > On Wed, Oct 10, 2007 at 09:51:35AM +0700, Patrick Shirkey wrote: > > > On Tue, 2007-10-09 at 09:16 -0400, Forest Bond wrote: > > > > On Tue, Oct 09, 2007 at 01:11:38PM +0200, Lars Luthman wrote: > > > > > Slightly-slower-hack: add the 'sync' option to whatever script or > > > > > config > > > > > file is used to mount the device (/etc/fstab, udev rules, KDE service > > > > > menu etc). That way all writes will be synchronised automatically. > > > > > > > > It sounds like this doesn't matter much for you, but keep in mind that > > > > -o > > > > sync doesn't cause reads to be unbuffered, so don't go trying to > > > > benchmark > > > > your flash devices that way! > > > > > > Hmm, I need speed and accuracy. > > > > But only for writes, no? > > For the purpose of this script I am happy with just write but I will be > needing read in the near future. > > FYI, I can connect 200 usb-storage devices to a dual cpu intel with 4 GB > RAM and an extra pci-usb card on the same irq as the internal usb bus. > > Basically I am uploading content to a couple of hundred mp3/multi media > players at the same time and this script is used by other people with > less Linux knowledge than I so I'm trying to make the progress bar as > accurate as possible. Otherwise there is a significant lag between when > the progress bar says it is finished to when the transfer is actually > finished. Speed is also a major issue here so I cannot use the sync > option as that causes way too much delay compared to async. > > Can anyone here shed a little more light on how to monitor the disk > activity while transferring data? if you use your own compiled code for the copy, you can open the destination files with O_SYNC. from man 2 open: O_SYNC The file is opened for synchronous I/O. Any write()s on the resulting file descriptor will block the calling process until the data has been physically written to the underlying hardware. See RESTRICTIONS below, though. this avoids having the entire mount be done with the sync option. ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
Re: [LAD] [OT] bash vs disk activity
> Can anyone here shed a little more light on how to monitor the disk > activity while transferring data? > Not sure if this would work, but first thing I can think of is: 1) Calculate the amount of data to transfer (du command.) 2) Find out the remaining free space in the target unit (df command.) 3) Calculate the remaining free space when the copy is done. 4) Begin copy. 5) Periodically monitor the free space. HTH, Luis ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo.cgi/linux-audio-dev
