I think your suggestion is very similar to my second method except that I give the task of deciding the type of the next message to the clients and you keep it internal to the PortMidi API. There is a problem with your idea which is that the state (sysex, non-sysex, empty) has to be updated asynchronously by the system thread and at least one client thread. There may be a lock-free way to do this, but it's not obvious. The second problem is that you can read both sysex and non-sysex data in a single PortMidi message: real-time messages (one byte) are allowed by the MIDI spec and by PortMidi to be embedded within sysex messages. This is why I think the best approach is have just one thread read everything and dole some of it out for further processing by another thread if necessary. If you're willing to let two threads be mutually dependent (sysex can't progress until non-sysex reads current non-sysex message, and non-sysex can't progress until sysex thread reads sysex message), then it can't be any worse to let one thread read everything and the other thread depend upon it to get the messages it wants.
-Roger Fabien Costantini wrote: > Thanks Roger, it does help. > > I'm unlucky that it seemed to work for me probably because I was scrutating > the fifo > faster enough not to miss events but now I understand > why my original approach doesn't work. > > I thought about the 2 solutions (fifo splitting and selective Pm_Read()) > you mentioned and agree that none of them is really satisfactory at the end. > > I believe that even if we would implement 2 fifos in portmidi, > this would solve my problem but would not be elegant. > > More precisely, it would not address the need of reading messages ordered, > which is sometimes needed in other midi applications. > What I mean by this is that int the 2 fifos solution ; it would be ordered > inside a category (non sysex data or sysex data), > but we may lose the link between the sequencing of all midi messages, > sysex or not. > > So I had a third idea that would need the help of portmidi developers, > but would be IMHO useful and polyvalent: > Let's suppose we still conserve a unique unified midi fifo, but that we have a > flag associated to the fifo we read from (as opposed to associated with the > device > like the async sysex_in_progress flag) which gives only the information > that what is to read currently in the fifo is either: > - nothing > - a sysex message > - a non sysex message > > This would be great as we would consume only what we need. > If a msg wouldn't be of our thread interest then it would unlock immediately > and would try later, meanwhile another thread would read these events. > What may be non trivial to determine for the client and that this new code > would do > is to achieve a correct response if a long sysex is currently available on > the fifo > but was splitted into several parts so that reading in the client of the > first message > would not give anymore the informaton of a sysex SOX for instance, > in the case of SOX/EOX sysex messages. > > In other words this flag would 'keep track' of previous messages states. > an API would permit to interrogate what is currently available like: > int Pm_Read_Message_Type() that would return the 3 possibilities mentioned > earlier. > > The usefulness of such a code addon would be multiple: > - use for selective multithreaded Pm_Read() > - use for splitting easily fifo in client code > > Any thoughts ? > > Thanks, > Fabien > > ------------------------------------------------------------------------------ > Roger wrote: > Fabien, > I may be missing something, but I think I understand what you want > to do, and I think it will not work. On the MIDI input side, PortMidi > delivers data from an input device to a FIFO using a system-provided > thread or callback. The client runs asynchronously to read from the > FIFO. Let's call these threads the "system" and the "client". The > sysex_in_progress thread is set/cleared by the system. Since this is > independent of the client, the system can begin processing a sysex > message, set sysex_in_progress, finish the sysex processing, and clear > sysex_in_progress, all before a client thread wakes up. > > There are a couple of ways to split sysex and non-sysex messages > safely. One method is to have the non-sysex client read everything. > When sysex messages are received, copy the data into (yet another) fifo > for the sysex client. (You'll probably want to copy the data into your > application's data structures and strip out any 1-byte real-time > messages, which PortMidi is allowed to embed in sysex messages, so by > folding in this formatting with the copy, this scheme should only add a > small amount of net overhead). > > A second method will require a mutex lock to call Pm_Read and a > structure of type PmEvent. Both the sysex and non_sysex threads are > free to read at any time, but if one accidentally reads a message of > the wrong type, the message is left in the PmEvent for the other > thread, and reading does not proceed until the other thread takes the > message. In psuedo-code, the non-sysex client looks like: > > > non_sysex_read() > get_the_lock > if pmevent.message == 0 // no data, look for some > if Pm_Read(stream, &pmevent, 1) <= 0 > release_the_lock > return "nothing -- try again later" > // message to handle is in pmevent > if pmevent has non-sysex data > result = pmevent > pmevent.message = 0 // clear the buffer > release_the_lock return result > else // need to wait for sysex thread to read this > release_the_lock > return "nothing -- try again later" > > The sysex_read looks the same except "pmevent has non-sysex data" is > replace by "pmevent has sysex data". This scheme suffers from the need > for locks, the possibility to have two threads busy-waiting on the > fifo, and the fact that the sysex thread is going to have to filter out > non-sysex realtime messages and do something with them. You could fix > the busy waiting with condition variables, but that's only one of the > problems. > > You *could* modify PortMidi to use separate FIFOs for sysex and > non-sysex data, and this would actually simplify the handling of > realtime messages embedded in sysex messages (the system thread could > simply distribute the bytes into the appropriate fifo). Some midi > systems do this, but it requires every client to poll two fifos instead > of one and it gives you two buffers to potentially overflow. > > I hope this helps. > > -Roger > > Fabien Costantini wrote: > I handle simultaneously several midi in/out sources in real time, > all those peripherals exchange standard midi events > as well as custom mixing automation events encapsulated in midi sysex in real > time. > I must re-route and treat some of these mixed events but different threads > treat sysex events and non sysex event. > It's a multiple producer /multiple consumer scheme, where consumers are > selective > (those consuming sysex don't consume non sysex data and vice versa). > I would be more than happy if I could select a different message queue (i.e: > 2 independant queues) for non sysex data > and sysex data but don't think it's possible with pm ? > So my solution based on 'listening' before consuming messages is fast and > easy to manage. > (I could have read the unified midi buffer an splitted it myself into 2 > sysex and non sysex queues > but it's more work and will slow down things unnecessarily) > > Thanks, > Fabien > > _______________________________________________ media_api mailing list [email protected] http://lists.create.ucsb.edu/mailman/listinfo/media_api
