Re: [RDD] rdimport - audio file location and ingestion
On Nov 7, 2018, at 10:03, Rob Landry <41001...@interpring.com> wrote: > A client of mine used to use AudioVault. I spent lots of hours examining AV > .wav files to see how the metadata was stored, and was surprised to discover > that each file stored its metadata twice: in an AV10 chunk, and again in a > CART chunk; Lord only knows why. I was left to guess which one was supposed > to be authoritative. The CART chunk is publicly documented as an AES standard [AES46-2002-(r2007)] and is there to facilitate interoperability with other systems (a Good Thing). The AV10 chunk is proprietary to AudioVault, and contains additional ‘secret sauce’ (things like outcue strings, post marker location, etc) that are not included in the standard CART chunk. (Rivendell BTW will recognize and use *both* when importing metadata from an AV system). Other vendors have other 'secret sauce’ chunks of their own that serve similar purposes, ‘SCOT’ and ‘ENCO’ being two that come to mind immediately (yes, Rivendell can read those too). > AudioVault struck me as a thinly disguised Pandora's box of chaos and > contradictions. There were layers on top of layers, each one seemingly > written by people who knew nothing about the others. Conway’s Law in action. [See https://en.wikipedia.org/wiki/Conway's_law] > Date and time formats varied from file to file; I found one in which the > durations of audio files were stored as floating point numbers representing > fractions of a day. My favorite is in a feature of the VMS operating system, where an operator could manually specify a system clock offset at boot time. The units of time used was the micro-fortnight. (I leave it as an exercise for the student to work out a conversion from seconds). > And underlying it all, like Satan in the pit of Dante's Inferno, was Windows, > an operating system written for secretaries and accountants. AVSAT would > crash from time to time during Boston Celtics games when it tried to record > news headlines into a file that AVAIR had cued up to play at the next break. > The Celtics would call for the break, AVAIR would play it, and then there'd > be silence... Rivendell would have handled it all seamlessly. No further comment necessary. Cheers! |--| | Frederick F. Gleason, Jr. | Chief Developer | | | Paravel Systems | |--| | There is only one thing worse than having your competitors trying | | to inter-operate with your systems - and that is to have your | | competitors *not* trying to inter-operate with your systems.| | --Alan(UK), GrokLaw.net| |--| ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] rdimport - audio file location and ingestion
On Tue, 6 Nov 2018, Fred Gleason wrote: Thus, in Rivendell, we make a strong separation: the audio store holds audio and the database holds metadata. Audio in the audio store is *immutable*; once it’s written, it can never be changed short of being *deleted*. Information in the database OTOH can be changed at any time. This separation immediately takes an entire class of hard-to-find-and-fix bugs involving the synchronization of two non-orthogonal processes off the table while greatly simplifying the conceptual model of how the overall system works: a major win in each case. That makes perfect sense. It also allows us to use an off-the-shelf RDBMS (MySQL/MariaDB) that has had hundreds of thousands of man-hours invested in its design and debugging instead of having to whomp up one of our own (a major savings of resources). While this does create two sets of data that each require backup, there are well-understood techniques in the IT industry for managing such situations. What I do for several stations is maintain a second Rivendell system at the transmitter site. I run a cron job at the studio that periodically compares the CART and CUTS tables between the studio and transmitter systems; it updates the database at the transmitter with any changes made at the studio, and rsync's the corresponding audio files from the stutio to the transmitter site. I have a second cron job running that does the same thing with logs. The transmitter site system is therefore up tyo date with the latest songs, spots, voice tracks, weather forecasts, etc. and can be put on the air with the flick of a relay if anything goes wrong at the studio. In addition, I have the MySQL database at the studio backup up nightly by yet another cron job, and the backups kept for a week before being overwritten. Rob -- Я там, где ребята толковые, Я там, где плакаты "Вперёд", Где песни рабочие новые Страна трудовая поёт. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
Re: [RDD] rdimport - audio file location and ingestion
Well, the other side of the coin is that if you have two copies of the metadata, one in a MySQL database and the other in the file headers, then you need some mechanism to keep them synchronized. Otherwise you become the proverbial man with two watches who can never be sure what time it is. A client of mine used to use AudioVault. I spent lots of hours examining AV .wav files to see how the metadata was stored, and was surprised to discover that each file stored its metadata twice: in an AV10 chunk, and again in a CART chunk; Lord only knows why. I was left to guess which one was supposed to be authoritative. AudioVault struck me as a thinly disguised Pandora's box of chaos and contradictions. There were layers on top of layers, each one seemingly written by people who knew nothing about the others. Date and time formats varied from file to file; I found one in which the durations of audio files were stored as floating point numbers representing fractions of a day. And underlying it all, like Satan in the pit of Dante's Inferno, was Windows, an operating system written for secretaries and accountants. AVSAT would crash from time to time during Boston Celtics games when it tried to record news headlines into a file that AVAIR had cued up to play at the next break. The Celtics would call for the break, AVAIR would play it, and then there'd be silence... Rivendell would have handled it all seamlessly. Rob -- Я там, где ребята толковые, Я там, где плакаты "Вперёд", Где песни рабочие новые Страна трудовая поёт. On Tue, 6 Nov 2018, Richard Elen wrote: IIRC, you can store a full complement of metadata in a WAV file and there is a standard for it, it's just not commonly done. I must admit that the one thing that has concerned me slightly is the fact that the ingested files are completely unidentifiable - the idea of writing out and maintaining the metadata in the ingested files in case of a need for database reconstruction sounds like a neat idea on the face of it. On Mon, 5 Nov 2018, Fred Gleason wrote: ...This was one of the fundamental design principles laid down at the very inception of the project: the audio store contains *only* audio; all other metadata goes in the SQL database. What was the rationale behind this decision, incidentally? Given the amount of thought that has gone into the project as a whole, I imagine there is a very good reason for it. -_R On 06-Nov-18 16:50, Rob Landry wrote: Of course, one consequence of this approach is that if you lose the MySQL database, the audio files will be more or less useless. If each audio file had headers carrying copies of the metadata, it would be possible to reconstruct the database fromt hem in the event of a disaster. ___ Rivendell-dev mailing list Rivendell-dev@lists.rivendellaudio.org http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
