On Nov 6, 2018, at 14:06, Richard Elen <re...@brideswell.com> wrote:
> IIRC, you can store a full complement of metadata in a WAV file and there is
> a standard for it, it's just not commonly done.
>
There are several public standards for such, as well as a whole slew of _ad
hoc_ private ones, invented by vendors for use in specific product lines.
> I must admit that the one thing that has concerned me slightly is the fact
> that the ingested files are completely unidentifiable - the idea of writing
> out and maintaining the metadata in the ingested files in case of a need for
> database reconstruction sounds like a neat idea on the face of it.
>
There are some options in Rivendell for aiding the recovery of a damaged audio
store. See for example the --relink-audio switch in rddbcheck(8).
> On Mon, 5 Nov 2018, Fred Gleason wrote:
>
>> ...This was one of the fundamental design principles laid down at
>> the very inception of the project: the audio store contains *only* audio;
>> all other metadata goes in the SQL database.
>
> What was the rationale behind this decision, incidentally? Given the amount
> of thought that has gone into the project as a whole, I imagine there is a
> very good reason for it.
At the time (ca. 2002) I was very familiar with a radio automation program
(name withheld here to protect the guilty) that used this very technique of
putting the metadata into the audio files. They had an auxiliary program called
the ‘SoundHound’ (derisively referred to as the ‘SoundPig’ by one of my
co-workers) that would continually scan the set of audio files on the system,
building and updating a searchable database of the metadata. For you see, a
broadcast automation system *needs* a database; it’s the only way you have of
finding things quickly enough to be useful in a near-realtime system. The
SoundHound would frequently get behind-hand in keeping the database in sync
with the metadata that was in the audio files, resulting in all sorts of races,
crashes and other system mayhem. Also, what’s often forgotten is that metadata
in a broadcast automation system is an inherently *dynamic* thing; constantly
being updated and changed. Since the audio files were also the metadata store,
that meant that they had to rewrite the audio file every time a piece of
metadata changed: a major non-orthogonality in the system.
Thus, in Rivendell, we make a strong separation: the audio store holds audio
and the database holds metadata. Audio in the audio store is *immutable*; once
it’s written, it can never be changed short of being *deleted*. Information in
the database OTOH can be changed at any time. This separation immediately takes
an entire class of hard-to-find-and-fix bugs involving the synchronization of
two non-orthogonal processes off the table while greatly simplifying the
conceptual model of how the overall system works: a major win in each case. It
also allows us to use an off-the-shelf RDBMS (MySQL/MariaDB) that has had
hundreds of thousands of man-hours invested in its design and debugging instead
of having to whomp up one of our own (a major savings of resources). While this
does create two sets of data that each require backup, there are
well-understood techniques in the IT industry for managing such situations.
Cheers!
|----------------------------------------------------------------------|
| Frederick F. Gleason, Jr. | Chief Developer |
| | Paravel Systems |
|----------------------------------------------------------------------|
| It's always good to take an orthogonal view of things. It |
| develops ideas. |
| -- Ken Thompson |
|----------------------------------------------------------------------|
_______________________________________________
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
