The partitioning is setup as per the default of the broadcast appliance install, which I think has the /var/snd seperated in some way from the rest, but regardless it is all on the same 6 disk raid 5 array of 10,000rpm ultra320scsi disks. the server hardware I am using is a little outdated I will admit, but we are a non com and there is no budget now for upgrades. I actually am moving to a slightly newer server, but it is still a scsi system, so ssd is out of the question, unless there is a pcix solution (which would require me to lose my hardware raid controller, but I could use linux software raid, though I am lost at that point). what about pxe to load the OS/Rivendell into a RAM disk, and leave the physical disks for /var/snd. again, i'd be lost at how to do this...
anyway we are consolidating our two stations to one side of the building, and I am building out a new(er) rivendell system so that on moving day I can just sync up the /var/snd and do a database dump and restore to the new system and have them up and running in the new studio, but for this to work ideally I need to be able to backup the old /var/snd while the old system is running. I think I have it going now though, knocked the bwlimit down to 500 and it is working...going to take forever though and still doesn't explain why it used to work fine without any of the nice, ionice, or bwlimit..... anyway, now would be the time to play with it since I have a good three weeks to complete the transition, and it is mostly done already, thoguh I want to burn it in for a week or so to know it is stable. Nathaniel C. Steele Assistant Chief Engineer/Technical Director WTRM-FM / TheCrossFM On 10/2/2012 8:32 PM, Gregg Wonderly wrote: > This sounds like paging/swapping based I/O delays. The head is seeking all > over the disk it seems. Is your /var/snd on the same filesystem as the > rivendell executables? It might be useful to try putting it on another disk > if possible. Also, in this day and age, there is, of course, serious speed > benefits of using solid state drives. Since rivendell is so small, putting > all of it onto a small SSD might help eliminate all of these "waiting" > issues. Rivendell is trying to provide a "real-time" like application on top > of a non-real-time environment, so if you don't have spare bandwidth on all > fronts of memory, disk and network, then there will be moments when things > get ugly. > > Gregg Wonderly > > On Oct 2, 2012, at 4:23 PM, Nathan Steele <[email protected]> > wrote: > >> I tried this: >> nice -n 19 ionice -c3 rsync -av --bwlimit=20000 >> >> the problem remained, I stood in the controll room and watched the >> rdairplay screen, it would freeze for several seconds, run a few >> seconds, and freeze again, this went on for 2 minutes then I went and >> killed the rsync by ctrl-c, went back to the control room and this >> behaviour continued, to the end of the the song, then there was a period >> of dead air, the next element ( a liner) was jumped (it was not a hard >> time event either) and the next song played, things returned to normal >> at this point. the whole time the screen was freezing, the song that was >> playing played fine. >> >> if I understood correctly, 20000 should be 20000 kB, so 20 mB/s, on a >> gigabit connection should not be a problem... >> what am I missing? >> >> I did notice that out of 4GB of RAM on the server, i had only ~300 mb >> free, but that seems it should be enough also. for curiosity though, how >> do I find what is using the memory and free some up if possible? >> >> Any suggestions where to look for my bottleneck? >> >> thanks. >> >> PS I tried the ionice and bwlimit on the box I run samba on and it >> worked well, this machine hosts a database for our timeclock software >> and the rsync backup made it grind to a halt and timeclock becam >> unresponsive, the machine was just a spare clunker we had when our old >> one died, and it is pretty low spec. these commands made the timeclock >> usable during the backup, sluggish, but useable. >> >> Nathaniel C. Steele >> Assistant Chief Engineer/Technical Director >> WTRM-FM / TheCrossFM >> >> On 10/2/2012 1:36 PM, James Harrison wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> Correspondingly, the ionice command may help. >>> >>> Cheers, >>> James Harrison >>> >>> On 02 October 2012 18:08:33, Stan Fotinos wrote: >>>> Hi Nathan >>>> >>>> Have you tried using the following option? --bwlimit= >>>> >>>> Stan >>>> >>>> On 2/10/12 11:26 PM, Nathan Steele wrote: >>>>> Hi all, >>>>> >>>>> Having a problem with rsync lately (don't recall having it before, but >>>>> not sure what changed, or maybe I just didn't notice). >>>>> >>>>> The background info: >>>>> Rivendell broadcast appliance on a server machine with a 6 drive raid 5 >>>>> (hardware controller) hosting mysql and /var/snd, and the delta 1010 >>>>> audio card, rivendell broadcast appliance on an airplay machine, CAE >>>>> running on the server (airplay runs on this machine but the audio comes >>>>> out the server soundcard). A network drive is mounted as /backup on the >>>>> server machine. the rivendell machines have gigabit nics and are on a >>>>> gigabit switch, the network drive is 100 megabit nic. >>>>> >>>>> The problem: >>>>> Running rsync -alv /var/snd /backup causes latency between the songs >>>>> playing out. the songs play fine once they start, but we get a few >>>>> seconds of dead air between songs, and it really throws the jocks off if >>>>> they are on air. >>>>> >>>>> niceing it didn't help at all nice -n 19 rsync -alv /var/snd /backup, >>>>> actually seemed to make it worse... >>>>> >>>>> I can't imagine the network is clogging....the cpu utilization seems to >>>>> be fine, as well as RAM....... >>>>> >>>>> Any Ideas? >>>>> >>>>> the purpose of course is to backup my audio. any suggestions? am I using >>>>> the wrong switches? should I be using a negative number for nice? (I >>>>> read the docs and it seemed like i wanted a positive number..) >>>>> >>>>> thanks, >>>>> >>>> >>>> _______________________________________________ >>>> Rivendell-dev mailing list >>>> [email protected] >>>> http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v2.0.17 (MingW32) >>> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ >>> >>> iEYEARECAAYFAlBrJjQACgkQ22kkGnnJQAxBTwCfTo1GVT5aPy23MmauYyyYkj0M >>> yK0AoJpj226/UfoDlXkljfrfxlQaI2VC >>> =xzYa >>> -----END PGP SIGNATURE----- >>> >>> _______________________________________________ >>> Rivendell-dev mailing list >>> [email protected] >>> http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev >>> >>> >>> >> _______________________________________________ >> Rivendell-dev mailing list >> [email protected] >> http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev > _______________________________________________ > Rivendell-dev mailing list > [email protected] > http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev > > > _______________________________________________ Rivendell-dev mailing list [email protected] http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev
