>OK. So say I have an application dealing with audio video and MIDI. Most >of the time the audio will be used for the clock providing 'performance >time'. Video and MIDI will have to be synced to this time.
in my experience (limited), its really hard to convince the video people to buy this. they seem to think that they super-dooper SMPTE timestamps on their video data is the master clock, and that audio has to sync to that :( > Scheduling video frames >and MIDI messages on this time base is not practical however. So a common >'wall clock' or 'system clock' is needed. This has to be an unadjusted >clock. >I don't know if CLOCK_MONOTONIC is supported yet within linux, but this not via POSIX. its available either via the RTC that exists in almost all computers on which Linux runs, or via the cycle counter on those architectures that support such a thing. >is what should be used I think to schedule MIDI messages and video >frames. (You could call this UST). sure. this is what MusE and SoftWerk and a few other Linux MIDI apps do for MIDI sequencing (where the RTC is used for the UST source). i don't know about video, but my impression is that video is much more like audio: its a streaming data flow, and once you have a "synchronous execution" model like JACK, the idea of "scheduling" video frames pretty much vanishes into the ether. the application still has to know when to play what, but the passage of time is marked purely by it being told to handle N msecs worth of processing. thats why in an SE system, i don't think you need an explicit clock for any streaming data. MIDI is different, because its not streamed - its an asynchronous data flow that can go from saturating the wire to having no data at all for extended periods of time and back again. video and audio are not like this, AFAIK. > So a mapping between >'performance/media time' (MSC) and the system time >(CLOCK_MONOTONIC/UST) is updated regularly. If all APIs supported >this common clock than synchronisation between the different kinds of >streams would be easier. For my MIDI API I don't want scheduling on a >tempo clock in the API. This should be done by the application I >think. So it will only support ordered scheduling on CLOCK_MONOTONIC >and flush/drain functionality. If the audio API would give me the >time in CLOCK_MONOTONIC on which a certain buffer (MSC or some other >clock) was performed, than MIDI could be synced to that fairly easily >without direct support in the MIDI API for the specific audio system. I think this is much harder than you think :) Most MIDI devices cannot tell you when MIDI data will be delivered to the wire. Even the at the hardware level, there is a (small) FIFO that buffers between the device driver and the UART chip. On decent hardware, this tends to be 8-16 bytes long, which represents 3-5 NoteOn/NoteOff messages and about a few msecs of real time. My own approach to this has tended to be to deliver data directly to the device driver (i.e. do not use a "sequencer" layer) at my best approximation to when it should be delivered to the wire, and assume that the device driver and the hardware will work hard to do the best they can. Notions of time are best handled entirely in an application, as far as i can tell. I don't want an external system to try to deliver my stuff "when it is supposed to" unless "when" is "right now". The system's sense of priorities and so forth might be very different to what I want. >I also think that MIDI should be kept seperate from audio (ALSA) as much as >possible. That a lot of soundcards have an onboard MIDI interface is more an >implementation detail IMHO. But ALSA is a MIDI API as well an audio API. it contains 2 different MIDI APIs: the sequencer, which is more like the one you're talking about, and the rawmidi API, which is what you'd probably base yours on top of. >> JACK has an interrupt-cycle-resolution UST source (at least when used >> the existing ALSA PCM driver). It could easily add the cycle counter, >> but that doesn't make any sense given the clock skew that I alluded to >> above. > >Can it give a mapping of cycle counter and the JACK media clock? it could do, but it doesn't. there is no portable way to read the cycle counter. the RTC is considerably more portable, but you need root or CAP_RESOURCE to set the frequency. --p
