Re: [abcusers] tuplet beaming
Phil Taylor wrote: > On 5 Dec 2004, at 19:30, RWW Taylor wrote: > >> The back-quote character appears on the standard Mac keyboard on the >> upper-leftmost key, above the tab very convenient. > > On my G4 PowerBook it's the key to the left of the Z, so I guess it's > not really standardised, even on the same platform. Actually, it's not a platform thing, it's a locale thing. On nearly every ANSI layout keyboard (prevalent in North America, and apparently also in China...), it's the key above the tab to the left of the number row. On a lot of ISO layout keyboards (used in the majority of places outside of North America...), you instead have an extra key in-between the shift and the Z which usually has that backquote, but not always. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp output data structure
Well, considering you really need to parse most fields *anyway* in order for the parser to have a context for parsing the actual tune data (for example, if Key is currently G, then the "F" note I just read is actually an F#...), I'm not sure it makes much sense to leave that decision up to the calling program. You're going to have the parsed data in a structure *somewhere* for the parser itself to access -- may as well pass back that structure as part of the overall output, and save the calling application extra work. Fields and settings the parser doesn't recognize *should* be passed back to the caller as text. For example, if that key field had a flag like: abc2dulcimer:tuning=DAD ...then the parser should pass back that part of the key field intact (or broken into tag name and tag value, or maybe even the name broken into app name and tag name) as a substructure of a key structure. Likewise for many of the Xcommands, whose scope really doesn't fall into the job of a basic ABC parser. But any field which the parser *should* know about should be parsed and the results passed back to the calling app. IMHO, -->Steve Bennett Remo D. wrote: > To avoid unnecessary work, I decided to provide separate functions to parse > the field bodies instead of parsing them anyway. > > In other words, you first receive the entire body of the field as a string > and then, if you are really really interested, you call a function like: > > k=abcKey(string); > > to really parse the field body, extract all the information and pack them > into k. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp proof of concept
>From my quick glance at this, I'm impressed -- I didn't think it was even possible to get close to using a xBNF based parser for ABC given the various ambiguities and weirdnesses in the different versions of the specs. Resolving most of these in the preprocessor is a nice way to avoid many of those problems... A couple of specific comments of things you should watch out for, based on areas I had specific problems with when doing my own parser: - The H: field is basically a way to associate an arbitrary text block with a tune -- I'd treat comments and even %% fields as part of it, along with any : fields which follow a continuation. - I'm not sure if the \% sequence to get a percent sign in the text is valid, since that would also be a valid way to continue a line. (See the example in section 2.2 of the 2.0 spec -- a comment may follow a continuation. And there's no place which says that the space in that example needs to be there...) You can accomplish the same thing by using "\d37" (See section 8.2), although it would be nice to extend the spec to have something more visually obvious. - The Key field in the current 2.0 spec (section 3.1.14) also allows for a key of "none", which I don't think you covered in your .def file. And the clef allows for a clef of "none" as well. Since a Key field can contain just a clef (and no key signature), this is a conflict, and I've suggested changing the spec to require "clef=none" to specify no clef, and "none" by itself will mean no key signature. - Clef and Voice specifications in the 2.0 spec (end of section 6) allow for space delimited program-specific specifiers in the form: = I don't think that's covered in your .def. - Also, the spec allows for additional arbitrary clef names in addition to the ones mentioned in the spec itself. And thus by extension there may be other single-word specifiers in voice fields as well. You got all of those I'm aware of, but missed some clefs used by abctab2ps. -->Steve Bennett Tom Satter wrote: > Atwood, Robert C said: > ... (snobal text removed) ... >> >> Any thoughts on using a tool like lex/yacc / flex/bison for parser >> generation? > > > For the parser that I am working on, I am using Python. I > have the complete parser working now, using simpleparse, which > uses an EBNF description as its input. I am currently working > on the API and filling in the data structures from the > callbacks. The basic structure of my parser is this: > > A) Run a preprocessor that does the following: > 1) Determine if I am looking at pre-2.0 ABC or not. > 2) Remove line continuations (doing it correctly depending > on the ABC version) > 3) Remove all % style comments (not %% directives) and store > them for later use (if needed). > 4) In pre-2.0 ABC files, if ! is used as a decoration marker > (!ff!), then change it to 2.0 syntax: +ff+. > 5) If an H: field is found in the header, then any lines that > follow it that do not start with '%%' or have ':' as the > second character will be changed to have 'H: ' before them. > (I believe that this is a problem in the 2.0 specification, > which says that the H: is a multiline header - it is > a royal pain to do this nicely in the BNF, do I handle > it here). > 6) The %% syntax is irrelevant with the new I: format. As > such, the %% syntax is removed and replaced with I: format. > B) Use an EBNF description to completely parse the file. I am > using an EBNF that I wrote that is based on the ABC 2.0 ABNF > written by Henrik Norbeck. I added a couple of things to the > EBNF to allow parsing older ABC files and corrected a number > of mistakes and places where the grammar was ambiguous. > C) The parser calls callbacks in python which I use to populate > the data structures. I am still working on the data structures > that I use. > > So far, I have been able to parse all of the ABC files in my > small test suite (about 10 ABC files, including a couple of > very large files). > > You can find the code as it stands right now at: > > http://www.purecode.com/~tsatter/python/ > > tom To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Call for an API
Phil Taylor wrote: > I'd like to keep the API as simple as possible. Pass in a pointer to a > string > containing one tune, get back a pointer to a linked list of structs > which is easy > to convert to a picture of the music (anything from postscript to a > bitmap) or > to sound (midi, Quicktime tune or whatever). The devil of course is in > the detail. I pretty much agree with this, although it's going to have to be slightly more complicated to allow for file fields outside of tunes. When I started my own parser, I had a routine to split a file into tune sections (starts with X: or T: and ends with a blank line, and takes into account continuations and H: fields), and global context sections (everything that isn't a tune...), and it could be broken down into two basic parse routines, roughly defined as follows: ABCContext *parse_context_section(NSString *context_string, ABCContext *previous_context) ABCTune *parse_tune_section(NSString *tune_string, ABCContext *tune_context) While in my case, the types were all Objective C classes (and things were considerably more complicated for a variety of reasons...), if you specified that ABCContext and ABCTune were structure types (each pointing to a hierarchy of linked structures), and NSString was just a char buffer, you'd have a pretty good ANSI C API. You can always add on OO wrappers, access APIs, format converters, etc. on top of that. But at the core, it boils down to passing in a string and a context, and getting back either a context or a tune structure pointer. I'd concentrate on the core functionality and worry about adding anything else later. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: Re[2]: [abcusers] spam
Thanks, I'll give that a try. -->Steve Bennett Tom Keays wrote: > Try Toby Rider <[EMAIL PROTECTED]> > > On Mon, 30 Aug 2004, Steven Bennett wrote: > >> Steve Bennett wrote: >> >>> Hmmn -- I know a way to test that. If you're seeing this message, than that >>> means my theory is probably correct, since the email address I'm using to >>> send this to is NOT subscribed to the list, and I'm sending the message to >>> the abcusers-list address. >> >> Well... *That* was an interesting experiment. Pity it actually worked. >> What was also interesting was how many of the list member's mail >> programs bounced the message (quite a few). And everyone's replies came to >> me directly instead of to the list. >> >> Phil -- you suggested sending an email to Toby... I'm happy to do that, but >> I have absolutely no idea where to send that. The only list manager info I >> have is the subscription website, and there's no contact information there. >> Do you know? To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: Re[2]: [abcusers] spam
Steve Bennett wrote: > Hmmn -- I know a way to test that. If you're seeing this message, than that > means my theory is probably correct, since the email address I'm using to > send this to is NOT subscribed to the list, and I'm sending the message to > the abcusers-list address. Well... *That* was an interesting experiment. Pity it actually worked. What was also interesting was how many of the list member's mail programs bounced the message (quite a few). And everyone's replies came to me directly instead of to the list. Phil -- you suggested sending an email to Toby... I'm happy to do that, but I have absolutely no idea where to send that. The only list manager info I have is the subscription website, and there's no contact information there. Do you know? -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp proof of concept
Christian M. Cepel wrote: > Steven Bennett wrote: > >> Objective-C was a big surprise to me when I was forced to learn it for a Mac >> programming contract. For a language which is basically standard C with a >> very small set of extensions to add OO support, it's both easy to use and >> surprisingly powerful. It took me only a couple of months to go from being >> a big C++ fan to being an even bigger Objective-C fan. I now use C++ only >> when forced to do so -- it's *so* limited and awkward in comparison. >> >> > Yes.. This is EXACTLY the post I was trying to remember. Thank you. > > So how about it guys... a consensus on Straight ANSI C, or Objective-C > for the parser. As much though I love and prefer Objective C, and would use it for my own projects, I'd still recommend straight ANSI C for this particular project, given it's stated goals. Mainly because Objective-C isn't very well known outside the Mac world, but also because there are runtime bindings (just like C++) which would have to be dealt with if you wanted to link Objective C into anything else. IMHO, -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp proof of concept
Jeff Szuhay wrote: >> Someone stated that using ANSI C would be best but that we would >> definitely want to use the object oriented extensions to make it object >> oriented C (not C++)... Perhaps that is ANSI C today... I dunno... I >> haven't programed in C for 5 years and perhaps ANSI has certified an >> updated C spec to include the object oriented routines/extensions. > > Uh... Objective-C? :-P > > (Oh, I couldn't help myself. You can slap me for that one), I wouldn't slap you for that -- I almost answered the same thing myself, but I suspect I would have meant it more seriously... Objective-C was a big surprise to me when I was forced to learn it for a Mac programming contract. For a language which is basically standard C with a very small set of extensions to add OO support, it's both easy to use and surprisingly powerful. It took me only a couple of months to go from being a big C++ fan to being an even bigger Objective-C fan. I now use C++ only when forced to do so -- it's *so* limited and awkward in comparison. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp proof of concept
Paul Rosen wrote: > This certainly seems like a universal problem, not just ours. I'm only > familiar with the Windows environment, and it is simple to make a DLL with a > straight C or COM or .NET interface that any other windows program can use. > Is there no mechanism like that on MAC and LINUX? .NET may eventually be portable to many systems through some open source projects out there. Personally I find .NET to be a poor design and too closely tied to C#. I won't use it and don't recommend it. If you want a good framework tied to a specific language, use Java or GNUStep/Cocoa instead. COM is MS only. CORBA might work, but I don't know much about it. Straight ANSI C in source form is probably the most portable thing out there, and in a statically linked library (for each platform) is a close followup, requiring only a recompile for each platform. Putting it into a DLLs or the like are also possible, but will require some platform specific code. > I'm thinking that there will probably have to be a portion of this that is > not portable. There will be the core parser that is portable and has all the > work in it, then there will be the interface, which may be different for > different languages and different OSes. If the parser is in ANSI C, and simply returns a pointer to a C data structure (which probably points to arrays and other structures...) containing the parsed data, then the interface is pretty simple and totally portable. You'll need to add some glue routines for various languages to convert that data into something the language can access, but that's not actually part of the parser itself. A while back, I suggested the returned data structures should be serialized into a buffer with all internal pointers converted to offsets, to make it easier for some languages. After some thought, I think the parser itself should just return the top level C structure pointer. Any other data format conversions - to a serialized buffer with offsets; to an intermediate text format; to another language specific format, etc... - can be separate glue routines. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABCp proof of concept
Remo D. wrote: > Hi there. I've prepared a proof of concept. It's only a scanner for ABC > files but already features Lua handlers (as well as C handlers). My biggest concern is that ABC (especially when you try to support older *and* newer versions, which is really necessary given how much stuff is out there) has enough special cases, state dependencies, and other oddities in it's parsing that I think your scanner will probably need a whole lot of context to work with. Certainly the handlers will need to have that context, and if you keep all context out of the scanner, the handlers will need a LOT of context. For that matter, I don't think your handler processing is going to work as it stands unless each handler contains a bit of code for each possible state a token could appear in -- which could make them huge, difficult to manage, and very difficult to extend. For example, take the token "C". Well, that could be a note, or it could be a key name, or a syllable in a lyric line, or a clef name, or a chord name, or a meter, or part of a title, area, book, composer, discography, or who knows what else. In a simple scanner as you're suggesting here, which doesn't look at the context, in most cases you're likely to have that come back as a T_NOTE token. Then in the handler for T_NOTE, it's going to have to figure out from the context what on earth it's going to do with this "C" token -- a fairly complicated task. The task becomes easier if you switch to a callback mechanism to obtain the next token (and one to push it back on the token list if you decide not to use it), and recursively pick your handler based on the current context. For example, the handleMeterToken handler would itself call abcNextToken() to read in the next token, and have it's own list of handlers for those tokens which might be relevant within that M: field. So the "C" token there would be processed by the handleTNoteInMeterField handler instead of something more generic. As far as using Lua for the handlers, I'm not certain what advantage that gains you if any, especially given how much context you'll need to be passing back and forth to the handlers, and the need to call back into C to get the next token. (Is that even possible in Lua?) I suspect the overhead of calling between Lua and C will slow the parser speed down considerably as well. If you're going to use Lua at all, I'd think you'd be better off writing the whole thing in Lua instead, leaving only a single C entry point to feed in a file and get back a data structure. Or better yet, forget the Lua handlers and write the whole thing in pure C. Lua isn't very well known (I first heard about it here...), and an application writer is more likely to pick code in a language he knows than one he doesn't. Paul Rosen wrote: > I'd define success as one or both of the following: > > 1) Everyone who is supporting ABC programs likes it and switches to this > parser so that every program will interpret files the same, now and forever. I'd undefine this one. I don't believe it will ever happen, simply because I don't think it is possible to define a single ABC parser capable of serving the needs of all applications out there. You can make one which will serve most of the needs of most of the applications, but it's never going to be perfect. Instead, I'd define a "success" goal of having a parser that is sufficiently easy to understand and well documented enough that someone who needs to either modify or extend it can easily do so, and that someone who needs write a new parser can use it as a reference. Paul Rosen also wrote: > I don't agree with your statement "To keep things simple I won't use C++ > with all his many unnecessary complications." What complications are you > thinking about? The advantages of object oriented programming are so strong > it seems silly to arbitrarily limit the development team. Can you even get a > C compiler anymore that doesn't support C++? It is simple to avoid name > mangling and v-tables in the API, if that is what you are afraid of. I think the biggest limitation of C++ is it's suitability as a linkable library. I've had trouble linking a C++ containing library with a pure C application (much less any other language), because the C application doesn't have the C++ run time initialization code. You can get around that problem in a variety of ways, but they're mostly compiler and platform specific and not very portable. Once you start working your way around the compiler issues and portability issues, you've basically recreated COM (or some similar object model) from scratch. That said, if you know of a portable way to have C++ embedded in a library that is linkable into applications in a portable, compiler independent way, I'd say it could work for a decent amount of the internals. I would prefer Objective-C (it's really ideal for this kind of work), but as I pointed out above with Lua, it's not as well known. (Except in t
Re: [abcusers] K:none
John Chambers wrote: > Steven Bennett writes: > | "K:none" is already defined in the ABC 2.0 draft spec, although there's a > | slight ambiguity in that spec, since "none" is also shorthand for > | "clef=none". When I implemented that section of my parser, I resolved that > | in favor of the key, and required the full "clef=none" if you want no clef. > > Sounds reasonable. K:none should mean that you don't want a key > signature; clef=none should mean that that you don't want a clef. I > use both in a few abc files I have that generate pages of blank > manuscript paper. I also made sure that "T:" alone works for a title, > since I don't want those on such pages, either. And all of these are > useful for producing musical fragments, as examples in a document. I believe I decided that T: was a valid title field as well -- some pieces simply don't have a title. > | "K:" by itself is not documented in ANY version of the ABC spec as a valid > | sequence, and cannot be assumed to work in any program. In my own parser, > | again, that would cause an error on the field, which would cause the field > | to be ignored (in an attempt to recover), which would then cause all kinds > | of havoc. Stick with "K:none" instead. > > What I think I've seen is a few comments to the effect that C is the > default key. Since most abc software requires a K line, the only > thing this could be talking about is a bare "K:" without any key > information. Of course, I'd prefer to say that "K:" means "K:none", > not "K:C" (or K:Am" or "K:Ddor", for the benefit of anyone who > understands the difference. ;-). Maybe we should add a few comments in the ABC 2.0 spec that discusses what the behavior of a blank field line should be. That way we don't get a zillion different behaviors. > This does remind me that I've got a few funny questions about why I > sometimes do things like putting "K:G" at the start of a tune and > then "K:Em" or "K:Ador" before a later phrase. They usually say that > the second K line doesn't change anything. It does, of course, but > how do you explain that simply to someone who doesn't already > understand it? I'll usually say something like "Well, the key does > change, and I thought it would be useful to document this, even > though it doesn't effect the printed output." But this doesn't seem > to be very convincing. Until recently, I would have found such fields confusing as well. But having recently absorbed a lot of chord theory, it becomes almost necessary to have such fields where the key actually does change, because the chord progressions for "K:G" and "K:Em" are different, even though the key notation looks the same on the staff. That's also why I now prefer to display the key as a text string above the staff in addition to the on-the-staff notation. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] K:none
"K:none" is already defined in the ABC 2.0 draft spec, although there's a slight ambiguity in that spec, since "none" is also shorthand for "clef=none". When I implemented that section of my parser, I resolved that in favor of the key, and required the full "clef=none" if you want no clef. "K:" by itself is not documented in ANY version of the ABC spec as a valid sequence, and cannot be assumed to work in any program. In my own parser, again, that would cause an error on the field, which would cause the field to be ignored (in an attempt to recover), which would then cause all kinds of havoc. Stick with "K:none" instead. -->Steve Bennett John Chambers wrote: > Atte asked: > | Some tunes are not in any specific key, but it seems abc doesn't have > | any way to notate this. If I use K:C the song is shown correctly, but > | this opviously is corrupted when transposing it. > | > | Is there a solution withinn the current abc definition that I overlooked > | or should we consider extending the K: to allow K:none? > > Some abc programs have supported this for several years already. I've > used it occasionally, though not very often. It's most useful for > notating fragments of music, for example in documentation and > tutorial docs. > > Implementing it is usually trivial, as it maps internally to the same > thing as K:C. The only real advantage of K:none is that it doesn't > match when you search for K:C (or K:Am or K:Ddor). Of course, just > "K:" alone also works for this. Most software should support that, > too, by mapping it to the same thing as K:C internally. > > To subscribe/unsubscribe, point your browser to: > http://www.tullochgorm.com/lists.html To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] On parsers again - Outlook & PHP
Jack Campin wrote: > The odd thing is, here am I, more than 30 years on, sitting at > a Power Mac 9600/200 with 384Mb of memory - whereas the 1130 had > 32Kb, I think, and presumably ran at a few thousand instructions > per second - but despite having a few gigabytes of software under > the table I couldn't do the same analysis now. I couldn't have > imagined there'd ever be a computer you couldn't run Fortran on. There are GNU compilers for various versions of Fortran which have been ported to the Mac. There are also commmercial compilers available, from IBM, Absoft, and NAG Software, and possibly others. Um... That's Mac OS X, of course. I just noticed you have a 9600, so I don't know if you're running OS X or not. Absoft's compiler is available for Mac OS 9 as well, though. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC parser output data structure.
I was going to let this idea die quietly... If we were talking about creating a data interchange format, I'd agree 100%. But we're talking about creating a general purpose front-end parser that can be linked into assorted ABC programs so they don't have to write their own parser. Having the output of that be a text format defeats the whole purpose, because each application would then need to write it's own parser for *that* format. It's been suggested that such a second-stage parser would be a trivial job to write. As a programmer who has written a number of parsers of various complexity over my career, and looking at the amount and complexity of the data we're talking about, I'd have to say that "trivial" is doubtful. It's probably not a complex task, assuming the text format is designed properly, is totally unambiguous, comprehensive, and flexible enough to do the job. But it's bound to be tedious and time consuming. I suggest anyone who thinks this is a trivial job should actually design such an intermediate text format, and then try and write a parser for that format. Let me know how long it takes. ;-) As a final thought - if anyone *really* wants to see text output from such a parser, let the parser be written to output C structures or whatever the programmers want, then they can write their own back end to the parser to convert the output into any text output format desired. It should be relatively easy to do. And everyone can be happy. -->Steve Bennett Jeff Szuhay wrote: > On Wednesday, July 28, 2004, at 03:43 pm, Bernard wrote: > [snip] >> The maximum is ascii. You can even read it without a computer. >> >> Flexibility is maximum in ascii. A new keyword is added and the old >> software doesn't understand the keyword and will ignore it. The >> problem of upgrading software is old software which won't read the new >> software's output at all. > > [snip] > > I agree completely with this. In fact, for the past 10 years, the whole > of computing has moved towards ascii-based (character-based) data > interchange standards away from binary data formats. To wit, SGML, HTML > and its variants, XML, as well as scripting languages which remain text > based and uncompiled (binary data). As an example, Perl, Python, > Javascript run on more platforms than I know. A powerful database > environment I worked in years go made every record available as text > (it lives on in Universe and > "multi-value" databases). LaTex and PostScript ... text based. > > The only (bad) reason to not use ascii for text based data IMNSHO is > when a vendor wants to maintain control of their proprietary data > format. Good for them but a real PITA for everyone else. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC parser output data structure.
Bernard wrote (portions snipped): > I disagree entirely on the "maximise portability". The maximum is ascii. You > can even read it without a computer. > ... > Sorry, but it seems archaic to me (in a situation such as we are talking > about) not to write the file in ascii. First off, we're talking about a general purpose parser here, not a file conversion program. In all likelihood, most, if not all, of the programs it will be used with will probably link directly to it, and there will be no intermediate file or storage involved - just a data buffer of some sort returned from a function call. The output isn't intended for an end user - it's intended for a program. The whole idea and purpose of a parser is to put the data in a machine readable format that the rest of the program can actually make use of in a non-sequential fashion. Even with ABC, most of the possible applications out there needs to maintain some information for non-sequential access. A notation program needs to access the key, clef, and time signature as it starts each staff. A player program needs to access the tempo, and will need to loop through repeated notes. Format conversion programs may need to access everything in a completely different order. An editor program might need to randomly access any individual range of notes. And so on. Not to mention that all of them need to access numerical values as actual numbers, not strings. Converting ABC into yet another ASCII based format means that you need to create yet another parser for *that* text format to get it into something your program can actually *use* in a non-sequential format. (And what would the output of *that* parser be, eh?) You could possibly write that secondary parser to simulate random access - a call to get the key of Tune 15 as of the 23rd measure, for example. But that would be prohibitively expensive in terms of processing time as you reparse the ASCII stuff over and over again. (Yes, even with today's faster processors...) Not to mention the complexity of writing functions to return every conceivable bit of data. Honestly, it makes absolutely no sense to me whatsoever to be making an intermediate ASCII text format the output of a general purpose parser. It defeats the entire purpose of there *being* a parser in the first place. Mind you, I agree that C structures are somewhat archaic, but they do the job. Ideally the output should be a set of objects, but then you'd lose cross-language portability. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC parser output data structure.
Paul Rosen wrote: > What projects are there? My imagination leads me to the following. I'd love > to know how else ABC could be used. > > Standard Notation > Tablature > MIDI Player > Transposer > Indexer > Pattern Matcher > Conversion to another format > Instruction (probably needing fragments) The obvious missing one that comes to mind is an Editor. I'd have a lot of trouble seeing how a general purpose parser as you've laid it out could work for an application like Barfly, for example. I have designed and partly wrote an Objective C based ABC parser (currently shelved indefinitely), and tried to make it general purpose enough to work for an editor app. Reworking my design for that was both a frustrating and educational experience, and gave me a great appreciation for what Phil Taylor has accomplished with Barfly. (Originally, I started writing my parser because I wanted four things: a notation based ABC editor, something which would generate PDF output directly instead of Postscript, something which would generate a new style of Hammered Dulcimer tablature I had worked out, and finally, something which would serve as a reference design. My Mac can now read Postscript files directly into Preview and convert them to PDF trivially, so I now use ABCM2PS for printing. My HD skills have passed beyond the point where I need tab anymore, but it would still be a good teaching tool. And what little ABC editing I do nowadays I've gotten used to doing with Barfly. So my motivations have waned considerably. But maybe I'll get back to it at some point because I still think it's a good reference work, and I have ideas for a player program... ;) -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC parser output data structure.
To add my two cents to this discussion: While I have my doubts that any single parser design could fit the needs of more than maybe half (being generous) of the possible applications out there, it's still a worthwhile project. I would avoid producing output in any format which needs further text parsing, such as a text or XML format. Which leaves a C structure-based design a good Least Common Denominator -- most languages can access C structures in some fashion. But the problem with C structures is that they aren't easily extendible, which makes it difficult to add features and change things. Another huge problem with C structures is that if they contain pointers, those pointers become invalid when the structure is moved through memory. (Such as might happen on some platforms inter-process communications, or when moved to another language.) However, with proper planning, you can make the returned format highly extensible and highly portable by taking a couple of steps early on: 1) Your parsing routine should return the data as a single linear buffer. While parsing, you can keep everything in separately allocated structures, but at the very end, when all the data is in structure form, you should calculate the size of the final output buffer, allocate space for it, and copy all the structure data into it. 2) There can be NO pointers in the buffer. Instead, you can specify offset values from the start of the buffer, which will serve the same purpose. Those values can be declared in structures as a pointer and used as such right up to the point where the structure is copied into the output buffer. Use this in combination with #1, and you become pretty much language agnostic. You can even turn the parser into a UNIX filter tool, or save the buffer as a file. 3) Place, at the beginning of each structure definition, a trio of variables which identify the type of structure, length of the structure, and a general purpose linked list pointer/offset value. This gives you extensibility -- the caller can simply ignore any structure it encounters whose type it doesn't know, and you can add fields to a structure later which callers can check for if the length is long enough. (In the past, I used to put a version number in the structure instead of a length, but I found that only rarely would I revise a structure -- I'd more often add a new structure type -- and almost always the revision would be to make it longer, so having the length was more convenient, especially when it came down to calculating the buffer size.) 4) Rather than putting a lot of data into a single large structure, break it up into many smaller chunks of data, each in it's own structure, in a linked list. This, in combination with #3, gives you a whole lot of flexibility and extensibility. (For example, all your tune background info -- history, author, copyright, discography, origin, etc... -- could be in a single linked list of structures pointed to by an "info" pointer/offset field in the Tune structure. And down the road, if someone, say, added an ABC field for Similar Tunes, you could just create another structure type and stick a copy of that structure in the "info" linked list.) 5) Avoid arrays of structures, or anything which assumes that structure Y follows structure X in the buffer. Every bit of data should be accessible by following a pointer/offset chain down from the first structure. Use the general purpose linked list pointer to group things together, and pointer/offsets inside the structures to point to other linked lists. You can use arrays of pointers/offsets as well, but the array should be fixed length. I've done this kind of thing on a number of projects (not ABC related), and it's always paid off nicely. I'm not saying you have to do it this way, but if you do, you will maximize your portability, and gain a lot of flexibility. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] 4 Days of absolute silence - AlphabetSoup
Christian M. Cepel wrote: > I had envisioned the parser working with a controlling program thus... > > o Program is responsible for helping the user to navigate the > filesystem (including url locations) to locate the file to be parsed. > > This creates two scenarios. > 1. In the case of a local file, the file is locked for access. File > reference is passed to the parser. > 2. In the case of a non local file, or an 'unsaved' buffer in the > program, a file reference is not appropriate. > > IMHO, memory is cheap, most every OS in wide use today supports some kind of virtual memory, and abc files are, in general, relatively miniscule. In fact, it wouldn't surprise me one bit if the data structure returned by the parser is significantly larger in memory use than the text fed into it. So I would recommend you avoid the file reference entirely and work only on a passed in memory buffer. You don't have to worry about it changing under you, or any file idiosyncrasies of the underlying system, and it will simplify your code quite a bit. The only environment where I see a potential issue with doing it that way might be a PDA trying to handle a Huge ABC collection file. But were I coding a parent program for such a beast, I'd write a brute force pre-parser to split the collection into individual tunes which get fed to the parser individually anyway. So again, the parser shouldn't have to worry about memory use. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Unicode [was: file names]
Christian M. Cepel wrote: > It was my understanding that all unicode character sets contain English > characters mapped to the same values they're mapped to in other sets. Close -- Unicode is a *single* character set. For convenience, you'll frequently run into references to Unicode code pages, but all they are is a range within the overall character set. All characters from every encoding that Unicode supports exist somewhere in that character set. So with a Unicode (UTF-8 or UTF-16) encoded text file you could easily have English, Chinese, Korean, Russian, and Symbol characters all in the same sentence. Another convenient item is that the first Unicode code page 0x0001 - 0x007f is the ASCII code. So if you're using wchar instead of char as your string pointer type, then comparisons like: if (str[0] == 'K') ...will work the same when using Unicode or ASCII. The only difference is now str points to an array of 16 bit values instead of 8 bit ones. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] File names
Phil Taylor wrote: > OK, I understand that. What was bothering me though, is how Steven B's > parser is going to deal with regular ascii strings which include a > space followed by a bracket. It's no problem when everything is > unicode, or everything is ascii, but if we are to have ascii abc which > may include unicode strings, we will need a way of indicating this to > the parser, will we not? Actually, you really can't mix encodings in the same file -- it will become nearly impossible to parse, since you have no way of telling where the switch occurs, and can't identify most encodings from their data anyway. The closest thing to a mixed encoding would be a UTF-8 file, which uses multi-byte sequences to reproduce Unicode characters which cannot be expressed in a ASCII. I'm not sure how the Unicode line endings I listed get translated into UTF-8 offhand. A Unicode text file (UTF-16 - each character is a 16 bit word, although Unicode has multi-word sequences as well...) is usually identified by it starting with the sequence 0xfeff or 0xfffe in the first two bytes of the file. If you're going to support Unicode, you should probably keep all your strings in unicode format internally, and convert everything else you read to that. UTF-8 is tougher. It has a similar identifying sequence (0xefbbff) but I've found not all UTF-8 files include that sequence. (And even some Mac tools forget to put it there. IMHO, they should fix that -- I may even report it as a bug...) If you're running on a system where UTF-8 is common, then you might try it with all files, since ASCII is a subset of UTF-8. Files in any other encoding (Old Mac OS multi-byte, Windows multi-byte, Shift-JIS, or any of the various 1 byte encoding pages, like Windows Latin-1 or Mac OS Western) are pretty much impossible to determine just from the file contents. If you really want to support them properly, you may need to have a means for the user to specify the encoding. The Mac OS X text editor lists some 84 different encodings that the user can tell it a file may be, and only 3 of them can be determined programmatically. I could add that kind of user setting to my parser, but I'm more or less satisfied to support the ones which are detected automatically - UTF-16, UTF-8 (if it has the proper identifier), and the user's local default encoding. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] File names
According to Apple docs (I'll take their word for it... ;): 0x2028 -- Unicode line separator 0x2029 -- Unicode paragraph separator -->Steve Bennett Stephen Kellett wrote: > In message <[EMAIL PROTECTED]>, Steven Bennett > <[EMAIL PROTECTED]> writes >> line endings needing to match throughout the file, and will accept Unicode >> line and paragraph endings > > What values (in hex - 0x, please) do those two characters have? > > Stephen To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] File names
> I'm just doublechecking since this conversation spun off of the > universal parser conversation... > > This conversation, while interesting doesn't actually pertain to the > parser does it? I've been trying to follow it in case it does. > > My understanding is that a parser will not do any file handling, and > that the super program calling it will do all the filehandleing. Is > this correct? That is mostly correct. The parser only deals with the file contents, not the file itself. That said, the file contents in the ABC specifications (including the 2.0 draft) are assumed to be strictly ASCII compliant, and I believe case sensitive everywhere. (Someone correct me if I missed something there.) However, string fields which are used intact and not parsed could conceivably be encoded with non-ASCII characters, with indeterminate results. The parser I've been working on extends this as a side effect of the string support in Cocoa. Since it reads and keeps all strings in Unicode internally (actually a simplification, but let it stand...), it will accept files in any text encoding that the system can correctly identify and will maintain all international strings. It will also be a bit less strict about line endings needing to match throughout the file, and will accept Unicode line and paragraph endings in addition to the usual CR/LF/CRLF. It would actually require a significant amount of extra code to defeat these features, so I'm accepting them even though they're not strictly conforming to the ABC specs. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] File names
John Chambers wrote: > Lest you think this is way off topic, I might mention that I've been > involved in attempts to use non-ASCII char sets in my ABC tunes. I > have a lot of "international folk dance" tunes, and it would be > really nice to be able to spell the titles right. Also, I like to use > single-tune files as my primary data (with little programs that > combine them for pages of tunes). It's really handy if the tune title > can be used in the file name. I've done this on my linux box, and at > least Latin-1 names work there. But when I rsync a directory over to > my Mac Powerbook, it goes berserk on the files with non-ASCII letters > in the names. > > This tells me that OSX "isn't ready for prime time" in the coming > international world. If it can't even handle a simple 'ä' or 'ö' in a > file name, how is it ever going to handle Chinese or Japanese file > names? It can't even handle a Finnish or Arabic file name. You can't > expect those people to use English file names. (Well, the Finns do > all speak English these days, but still ... ;-) Snip... > One question for our Scandinavian friends: Do any of you use Macs? > Can you get filenames that contain the non-ASCII letters in your > alphabet? If so, how do you make it work right? I've tried setting my > charsets to 8859-1 and UTF-8 and others, and none of them seem to > make the files in my .../Scand/ directory copy correctly from my > linux box. Copying between linux to this FreeBSD system works fine, > because those systems treat a character as unanalyzed bits. But when > copying to OSX, those files end up with gibberish names. Mac OS X has full support for Unicode, although not all the BSD UNIX utilities which have been ported over support Unicode to it's fullest extent, so there are oddities when you use the command line. That said, of all the systems I've ever programmed on, the Mac has the best international support of any of them -- internationalization has been a strong point for Macs since the early days. Apple's HFS+ file system (the Mac OS X default file system) stores filenames in UTF-16 Unicode format. This means I can (and do) have files with names in just about any language using just about any characters from anywhere in the Unicode code set. (Including mixing and matching entirely different language sets). What happens to those when transferred to a Linux system or a Windows system, who knows. The problem you're seeing is not that the Mac doesn't support internationalization, it's that it doesn't have any way of telling what the encoding is for the filenames you're giving it. Most filesystems out there (with a couple exceptions, like HFS+ and NTFS, which store filenames in UTF-16) encode filenames in some 8 bit string. To get international filenames, they use either different charsets, or UTF-8. But there's *nothing* in the filesystem itself which says "this filename is encoded in format ". That information is stored in the OS application layer as a *display* parameter. So it all looks correct on that system, because the OS translates it into the right characters when they get displayed. But when you try to transfer it to another system, all it knows is that the file is named some weird 8 bit string. This is why it gets all mangled in the translation. It's even worse when you send it via email, because you have to hope the email programs on both sides know how to deal with the encodings you are sending. I suspect rsync is the culprit in your case -- I seriously doubt that's been made Unicode aware. Probably your safest bet for a valid transfer is to burn the files to a CD using ISO-9660 format. There *is* a standard for filenames stored like this, that most systems ought to be able to read. It also should be possible to write a simple Demangle application which would read in a filename (or a directory of filenames), and given an encoding specified by the user, would translate it to Unicode and rename the file appropriately. Shouldn't be too complicated to write -- the standard OSX string routines have all kinds of support for translating strings between various encodings. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] reusable parser
> On Tue, Apr 27, 2004 at 12:01:00PM -0400, Steven Bennett wrote: >> Note that while ABC 1.6 and 1.7.6 explicitly allow these fields in-between >> tunes, ABC 2.0 draft states they can only be at the beginning of a file. >> (There really ought to be a note about this in the Deprecated Syntax >> section... Or the restriction should be lifted.) > > Yes. One consequence of this restriction is that abc files can become > "invalid" just by being cut&pasted together. If you have 2 files, each > with file-scope fields at the top, a simple > cat file1.abc file2.abc > file12.abc > will produce a file with headers between tunes. Forbidding this may make > life easier for simple parsers, but I don't think it's desirable for > users. Actually, that's an issue in all versions of ABC, because the file-scope fields at the top of file1.abc that are not overridden in the header of file2.abc would then apply to all the tunes in file2.abc, even if that would be inappropriate. I actually now believe (having written that part of the parser) that what should be in ABC 2 is either NO file-scope fields allowed, or they should be allowed anywhere outside a tune. The halfway measure of allowing them only at the start of the file can make parsers slightly more simple, but doesn't really simplify things all *that* much, and not at all unless you are parsing *strict* ABC 2.0 only. NO file-scope fields would be the ideal solution, letting you split and merge tune collections seamlessly. But since I doubt that would be accepted (and still has the issue of parsing pre-ABC 2.0 files anyway), I suggest we let those file-scope fields be anywhere outside tunes. IMHO, -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] !Current specification!
> In message <[EMAIL PROTECTED]>, Steven Bennett > <[EMAIL PROTECTED]> writes >> You know, it's amazing that people still have this silly impression that >> just because Apple only ships a mouse with one button, that the OS can only > > You learn something every day :-) I don't know that its a silly > impression - blame Apple's marketeers and Steve Jobs' reality distortion > field :-). I know some die hard Mac fans (one even owns a Lisa as well > as various Macs as part of his collection) and they haven't told me > this. Have all Macs been able to have multi-mouse buttons or only the > recent OS-X boxes? Mac OS X supports the right mouse button and scroll wheel natively pretty much everywhere, and applications may make use of additional buttons, although Apple software itself didn't have any use for them until Mac OS X 10.3. The right mouse button, as with windows, brings up a contextual menu. Mac OS 9 had support for the right button to pop up the contextual menu. I *think* it supported right buttons natively on USB mice, but can't be sure - definitely if you had a driver to generate the right event from the device, it would work, though. Third party mouse drivers have had support for extra buttons on the Mac since the late 1980s. Mostly things like doing double clicks and click-locks, although they also allowed programming buttons to do all kinds of wild and crazy things. > Thats one of my two dislikes of Macs done away with. The other one is > the menuing system. Click and hold is bad for anyone with WRULD / RSI. > The Windows Click, release, mouse/key around as you please then click > when you are ready is good for RSI. Can you configure the Mac to have > menus that are good for you rather than bad for you? Again, an old issue, long since fixed in Mac OS X -- you can click and release on the menu bar and the menu will stay popped up until you select from it. (And you can click and drag as well... Or click and release then use arrow keys.) Older Mac systems had it too, called Sticky Menus (I forget if this appeared originally in Mac OS 8 or 9), and again, third parties have had this fixed for ages. >> Amigas -- 4000/2000/1000, 2 C-64s, a TI-99/4A, and a SWTPC 6800 that are all > > TI-99/4A, now that is going back a long way. 16 bit, but registers in > RAM, not on the CPU, weird beast. Never got to use one. Amiga - no > budget, I had to settle for an Atari ST :-) Bizzarely enough they are > still trying to resurrect Amiga (gets mentioned on Slashdot from time to > time). I know. Alas, too little and way too late... -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] reusable parser
John Chambers wrote: > Yeah, but you could argue that it's not as big a problem with > Windows, because Windows (and MSDOS) is a separate OS that is its own > "standard" and has never been even minimally compatible with any > other system. People expect that porting software to Windows will be > a big deal, and will require a lot of rewriting. > > OSX is, however, a variant of unix. Much unix software runs on it > without problems. They even seem to have fixed most of the problems > with the aberrant CR line terminators, and switched over to the > standard LF terminators. And OSX is pushed as a member of the unix > family of systems. So you start using it, and discover this one > really nasty little gotcha ... Actually, back in 1982 when I first learned UNIX (actually Idris, the first commercial UNIX lookalike...), it was pounded into me that one should never take the case-dependence of the file system for granted, since there were *already* case-independent file systems out there that could be accessed with the right drivers. (Admittedly, those file systems were mostly on *floppy* disks, but the basic concept is still the same... ;) This was reinforced on the C side of things (which I was learning at the same time), because assuming case dependencies in filenames were considered non-portable in any case. (And back then, writing portable code was considered by us to be even more important than today...) Yes, the case independence of HFS+ (the default Mac OS X filesystem) is an issue if you got in the habit of naming two files the same except for case, but, IMHO, that has ALWAYS been a bad UNIX practice. If you never got into that habit, you shouldn't run into any problems on the Mac. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] reusable parser
> Here's a compromise: perhaps the parser can have a function like the above > that takes only one tune, that is, takes a string that starts with "X:", and > ends just before the next "X:" command. Then there would be a super-parser > (trivial to write) that breaks the string on "X:" boundaries. I'm guessing > that most applications would be concerned with just one tune at a time, > anyway. Actually, this is sort of close to what my parser is doing, but you're missing one *very* important thing -- the file fields. At the beginning of the file (prior to the first X: or T:) and in-between tunes (ie. After the first blank line in a tune, which ends the tune, and before the next X: or T:) there may be a variety of settings which can affect the remaining tunes in the file. In the ABC spec, these are the fields marked "Yes" for "File". Their existence complicates the job considerably. Note that while ABC 1.6 and 1.7.6 explicitly allow these fields in-between tunes, ABC 2.0 draft states they can only be at the beginning of a file. (There really ought to be a note about this in the Deprecated Syntax section... Or the restriction should be lifted.) You have four main approaches for dealing with these: 1) Parse the entire file sequentially. 2) Break it up into individual tunes, and ignore all the file fields 3) Break it up into individual tunes, and only allow file fields at the start of the file. (ie. ABC 2.0 style) 4) Break it up into individual tunes, but still support the file fields even if they're in-between tunes. I chose the 4th option. What I do is have my first pass parser break the file up into two types of section - global context sections (ie. Anything that isn't a tune - which includes file fields...) and tune sections. The first global context section is then parsed, creating a context object. That object is cached, and also passed to the parser for the next global context section repeatedly, resulting in a set of context objects, each of which corresponds to the context at the end of that global context section. This actually happens fairly quickly for most tune files, since there are frequently NO global context sections, or the only one found is at the beginning of the file. But I have the whole procedure anyway because the spec allows for these fields anywhere in-between tunes. Then, when I go to parse any given tune section, I pass in the context object for the last global context section that precedes the tune in the file. One other thing to be careful of in your first pass parser, should you go this route, is handling ABC 2.0 style continuation lines, and dealing with multi-line History fields. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] !Current specification!
> In message <[EMAIL PROTECTED]>, Steven Bennett > <[EMAIL PROTECTED]> writes >> Actually, I use Objective C when programming on a Mac. > > That explains my lack of awareness then. I don't use Macs and generally > don't like them either. Windows or Unix fine, but don't make me use a > computer with one mouse button. You know, it's amazing that people still have this silly impression that just because Apple only ships a mouse with one button, that the OS can only use one button, something which hasn't been true for many years. I've been using multi-button mice on Macs since the late 80s. My main mouse (actually a trackball) has 4 buttons plus a scroll wheel. *All* of them are useful out of the box with the OS. With the driver software installed, they're even more useful. I program and use Windows (98/ME/2K/XP) and Mac OS X for work, and at home I have Windows 2K, Mac OS X, and Linux machines. (Okay, that's only the actively used machines -- I also have a Windows 98 box, a Win95 box, three Amigas -- 4000/2000/1000, 2 C-64s, a TI-99/4A, and a SWTPC 6800 that are all mostly retired...) The Mac OS X machines are by far my favorites of the lot. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] !Current specification!
> In message <[EMAIL PROTECTED]>, Steven Bennett > <[EMAIL PROTECTED]> writes >> That it's written in Objective C should be fairly irrelevant, BTW -- it's >> almost trivial to pick up and understand the Objective C extensions to >> ANSI-C, and the resulting code is much easier both to write and read than >> either C or C++ would be. IMHO. > > You must be a rarity. Objective C. I've yet to actually meet anyone > thats used this language. Objective C, from what I've read is VHS's > Betamax. Better, but didn't succeed in the marketplace. Actually, I use Objective C when programming on a Mac. Because of the Cocoa framework there, it's *very* popular amongst Mac programmers, and is hands down the best programming choice for Mac applications. > Relevant or irrelevant, that is the #pragma? Its only irrelevant if > there are compiler implementations that all contributors can use on the > various platforms people want. Is there a GNU Objective C or equivalent? > > Don't take this an attack on your work or use of Objective C. Its not, > I'm just unaware of any Objective C compilers people can use without > having to buy yet another tool (something I can't see people doing when > contributing to a free project). Of course, if you can show me I'm > wrong, that'll be fine. Yes, GCC can compile Objective C. (And Objective C++, as well) In fact, GCC is the compiler used in the Apple provided developer tools on Mac OS X. You do, however, need to have some libraries, however, to do nearly anything. (Much like you'd want the standard C libraries for C, or STL for C++...) Fortunately, there's also an open source implementation of much of the Mac OS X Cocoa libraries called GNUStep which lets you write to those libraries on Linux, UNIX, BSD, and to some extent, Windows 2000/XP based systems. If you're not using the windowing/graphics system, it should be entirely portable. If you're using the windowing/graphics system, it's probably 95% portable to Linux, and about 85% portable to Windows. My parser will be using the non-graphic parts of Cocoa & GNUStep, so it should be highly portable. The printing code I'm writing, on the other hand, will necessarily use graphics, but I'm hoping it will work on most (and maybe all) systems. On the Mac, at least, it will output PDF natively - I'm not sure on the other platforms what the output will be yet. > Assuming there are Objective C compilers around that people can use that > brings up the question of language binding. How straightforward is it to > produce a C/C++ callable API for your Objective C created code? (If you > can do it for C/C++ we can sort out the Java/.net/Delphi/Python etc > straightforwardly). Objective C libraries are effectively the same as C libraries. Not too surprising since Objective C is mainly turned into C during preprocessing. With a tiny bit of glue, you should be callable from nearly anything. I plan to write that glue and sample code from C and C++ on several platforms. Not that being a plug-in parser was a design goal. It should certainly be usable that way (for nearly any currently popular environment except Windows 9x/ME -- Gnu), but as I said, it's intended as a reference work - to show *how* to parse ABC in a clearly understandable fashion. I hope people will look at it at some point, not so much to use it intact (it is, after all, not going to be efficient by it's nature), but as a guide to how to write their own parser to fit *their* needs. Which are no doubt different than mine. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] !Current specification!
> Why only writing a parser for one language (either C, C++, C# or Java)? > Surely it should be possible to generate a parser for any language from > a BNF grammar specification using a parser generator like SLK? > Incidentally, SLK is available for both Win32 as Linux. If ABC could be specified strictly as a BNF grammar, then I'd agree with you. Alas, there are so many exceptions and special cases (even in 2.0...) that I doubt there will ever be a totally complete and accurate BNF grammar for it. (Although Henrik Norbeck has done a very good job of approaching something relatively close to it...) That said, the parser I've been working on for the last few months has a high priority design goal of being usable as a reference work in the way described. I should warn you, however, that I quickly discovered that usefulness as a reference work is pretty much diametrically opposed to speed and efficiency in the code -- this thing is going to run slow and use a lot of memory when it's done. (Who knows when? ;) But pretty much anyone looking at the code should be able to easily see not only *how* it parses ABC, but *why* it is done in a given way. (And it will be commented *very* thoroughly...) That it's written in Objective C should be fairly irrelevant, BTW -- it's almost trivial to pick up and understand the Objective C extensions to ANSI-C, and the resulting code is much easier both to write and read than either C or C++ would be. IMHO. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Clef specification in 2.0
Hi all! I've been working (ever so slowly) on my ABC parser, and one item which came up was the specification of the clef field as listed in 2.0 spec. (Note that this message is 2.0 specific -- I'm aware there are other issues in what various pre-2.0 programs did that may be relevant, but that's another discussion...) It's mostly fairly clear, except for one area which I see as a potentially serious parsing problem. Specifically, it mentions that "clef=" may be omitted, but then later it says that "More clef names may be allowed in the future, therefore unknown names should be ignored." What bothers me is that means you could have pretty much any space delimited word mixed into a Key or Voice line, and not know if it meant a clef or something else. I'd like to see it changed so that "clef=" is required for any clef name *except* for "treble", "bass", "tenor", "alto", and "perc". Note I didn't specify "none" in that list -- that's because "none" by itself can be specified as the key signature, making "K:none" be ambiguous - does it mean a key signature of none, or no clef? (This was what originally triggered my examination of this part of the spec...) So I think "none" requires the "clef=" to be unambiguous. Given the added concept that the clef data may include pretty much anything in the form "=", I suggest we might change section 6 to read something like this: 6. Clef Properties Any number of Clef Properties may be provided in any combination in K: and V: fields, separated by whitespace. All clef properties are strings which follow the normal property syntax: = ...where propertyname and value may be any strings not containing whitespace. The standard clef properties include: clef=[][+8 | -8] - Specifies the clef for the current staff. May be "treble", "alto", "tenor", "bass", "perc" or "none". "perc" selects the drum clef. Other names (that do not contain digits or the '+'/'-' characters) may be used as well -- if your program does not recognize the clef name, it should use the default (treble) clef. As a special case, the "treble", "alto", "tenor", "bass", and "perc" clef names can be used as abbreviations for "clef=treble", "clef=alto", "clef=tenor", "clef=bass", and "clef=perc", respectively. The "none" clef name may NOT be used in this fashion. , if specified, indicates on which staff line the base clef is written. Defaults are: treble: 2; alto: 3; tenor: 4; bass: 4. This is useful when notating other common clefs. For example: clef=bass3 -- for the baritone clef clef=alto2 -- for the mezzosoprano clef clef=alto1 -- for the soprano clef +8 | -8 - draws '8' either above or below the staff. The player will transpose the notes one octave higher or lower. middle= - is an alternate way to define the line number of the clef. The pitch indicates what note is displayed on the 3rd line of the staff. Defaults are: treble: B; alto: C; tenor: A,; bass: D,; none: B. "middle=" may be abbreviated as "m=". This specifier can be handy when working in the bass clef. Setting K: clef=bass middle=d will save you from adding comma specifiers to the notes. transpose= - When playing, transpose the current voice by the indicated amount of semitones. This does not affect the printed score. Default is 0. "transpose=" may be abbreviated as "t=". The transpose specifier is useful for e.g. a Bb clarinet, for which the music is written in the key of C, although the instrument plays it in the key of Bb: [V:Clarinet] [K:C transpose=-2] stafflines= - The number of lines in the staff. Default is 5. Note that although this standard supports the drum clef, there is currently no support for special percussion notes. Applications may introduce their own properties. Their property name should start with the name of the application, followed a colon, followed by the name of the specifier. For example: V:p1 clef=perc stafflines=3 m=C mozart:noteC=snare-drum Applications which encounter a property they do not recognize should ignore it. === The other area I'd like some clarification of related to clef properties is in the K: line. First, where can clef properties can occur? Is it possible to have clef properties *before* a key signature, ie.: K:clef=treble Dm ...or must the key come first, if it's present? If the key can come second, is it possible to have properties before *and* after, ie.: K:clef=treble C transpose=-2 IMHO, I'd prefer limiting it so that the key would always come first if present. Finally, what key should be used if the first (and perhaps only) K: line in a tune doesn't contain a key, only clef properties? Or should we require that the first K: line must have a key (or "none") specified? Thanks, -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tulloc
Re: [abcusers] ABC and MusicXML
Jeff Szuhay wrote:
> Try looking at LilyPond.
>
> On Thursday, March 25, 2004, at 04:31 pm, Steven Bennett wrote:
>
>> I've often considered the idea of designing a more structured variant
>> of ABC
>> that would still be easy to read and write, but at the same be more
>> flexible, expandable, and more computer friendly. I pretty much know
>> exactly what needs to be done, but need to find time to write it up and
>> create a sample parser. One of these years I may get around to doing
>> just
>> that... :)
I have looked at LilyPond in the past, and in fact it's got some definite
similarities to what I have in mind. But it is far more verbose, lacks
human visual clues that would aid in reading it live, and overall shifts a
bit too much towards the computer friendly side, to the detriment of the
human friendly side.
My own concept would be much more similar to current ABC with the goal that
someone who can read existing ABC ought to be able to read the structured
variant almost intuitively. The main difference between what we have now
and what I would use would be enclosing everything in a construct which
might look like this:
{:}
...and completely ignoring end of line characters as simply being another
form of whitespace (which is mostly meaningless). Fieldname could be
anything that doesn't contain a curly brace, aren't limited to one
character, and may be abbreviated. Fielddata contains whatever contents the
field supports, and could contain subfields surrounded by curly braces, or
curly braces in the field itself escaped by a "\".
Thus, something simple might be:
{Tune:
{X: 3}
{T: Mary had a little lamb}
{M: 4/4}
{K: C}
{V1:edcd | eee2 |
ddd2 | egg2 |*}
{V1:edcd | |
dded | c4 ||*}
}
The staff break would happen after the fourth and eighth measures. But
basically by putting in this kind of simple structuring, you can keep the
file quite readable but make it far more parsable. The same tune could be
written:
{Tune:{X:4}{T: Mary had a little lamb}{M:
4/4}{K:C}{V1:edcd|eee2|ddd2|egg2|*edcd||dded|c4||*}}
...and it would still parse identically. And you could do things like:
{ Tune :
{ X : 5 }
{ Title : Mary had a little lamb \{round\}}
{ sabc2pdf/StaffSpacing : x3 }
{ M : 4/4 }
{ K : C }
{ P1 :
Melody
{ Voice1: |: e d c d | e e e2| }
{ Lyric1:Ma-ry had a lit-tle lamb }
{ Voice1:d d d2 | e g g2|* }
{ Lyric1:lit-tle lamb lit-tle lamb }
{ V1: |: e d c d | e e ee| }
{ L1:Ma-ry had a lit-tle lamb whose }
{ V1:e d c d | c4 :||* }
{ L1:fleece was white as snow }
} % End P1
{ P2 :
Round {
V1 :
z4|z4|
|: edcd | eee2 | ddd2 | egg2 | *
edcd | | dded | c4 :|| *
} % End V1
} % End P2
} % End Tune
...which isn't *quite* right but gets the concept (and a bunch of other
ideas, like program specific extensions) across.
Anyway, while I haven't worked out all the details (and who knows if or when
I will), I think the basic idea is fairly sound, and would add a whole lot
of computer friendliness with minimal cost to the human friendliness of ABC.
And with that, I go on vacation for a week...
IMHO,
-->Steve Bennett
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC and MusicXML
Richard Robinson wrote: > > Is anybody else here looking much at MusicXML ? I've been having a look > over the last few days, and I must say, I'm rather impressed. It seems > to me that this could all be tremendously useful to us, as ABC users. I looked at it and can see a number of places where it could be useful as a computer music storage and interchange format, but I don't see it as something that fits my own needs. It seems mostly useful as a intermediate step when converting from one music format to another - and not as useful as a native format, IMHO. Nobody is likely to write in MusicXML directly -- it's almost certainly going to either be a conversion from some other format, or someone using a computer program to enter notation. MusicXML is kind of the opposite end of the spectrum from ABC -- it's computer friendly, but not very human friendly, whereas ABC is very human friendly, but not so much computer friendly. I've often considered the idea of designing a more structured variant of ABC that would still be easy to read and write, but at the same be more flexible, expandable, and more computer friendly. I pretty much know exactly what needs to be done, but need to find time to write it up and create a sample parser. One of these years I may get around to doing just that... :) IMHO, -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: ABC 1.x continuations
> Steven Bennett writes: > | It was an outrageous example on purpose. It's *definitely* not legal ABC > | 2.0, but the definition I was hearing implied that it would be legal ABC > | 1.*. Which I didn't think it *ought* to be. Which is why I offered my own > | definition of what the actual 1.* behavior ought to be. > | > | That said, the following is perfectly legal in ABC 2.0, by the definition > | currently in the August draft spec: > | > | X:1 > | T:some made up tune > | M:4/4 > | K:Dminor > | abcd|efga|[K:\ > | G][M:3/4]def|gab| > | > | It's just not legal in ABC 1.*. IMHO. > > Well, I'd agree that it's legal ABC 2.0, but I'd also claim that it > should work under the earlier standards. I don't see any reasonable > way that a parser would classify the last line as anything other than > "music", which is the same type as the continued line. It's not a > "header" line, because it doesn't have a ':' in column 2. It doesn't > start with '%'. What else could it be except music? The problem I have with considering that as legal, is that it could just as easily have been written like this: X:1 T:some made up tune M:4/4 K:Dminor abcd|efga|[\ K:G][M:3/4]def|gab| If the one is legal in 1.*, the other should be. But this second one is a whole lot more ambiguous for a parser. It's just as well if we avoid the problem by treating *both* as illegal in 1.*, since it's unlikely anyone would actually want to use either form. And by doing so, you drastically simplify the parsing process. (Okay... You drastically simplify *my* parsing process.Someone using a different approach might find what you are suggesting easier to deal with...) -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: ABC 1.x continuations
It was an outrageous example on purpose. It's *definitely* not legal ABC 2.0, but the definition I was hearing implied that it would be legal ABC 1.*. Which I didn't think it *ought* to be. Which is why I offered my own definition of what the actual 1.* behavior ought to be. That said, the following is perfectly legal in ABC 2.0, by the definition currently in the August draft spec: X:1 T:some made up tune M:4/4 K:Dminor abcd|efga|[K:\ G][M:3/4]def|gab| It's just not legal in ABC 1.*. IMHO. -->Steve Bennett > On 16 Mar 2004, at 14:52, I. Oppenheim wrote: > >> On Sun, 14 Mar 2004, Phil Taylor wrote: >> >>> It's a pretty outrageous example. I don't think that parsers should >>> have to deal with continuations in the middle of inline fields, let >>> alone an example with another (non-inline) field inserted in the >>> middle. >> >> In ABC 2.0, continuations and ordinary comments will >> typically be dealt with by the scanner, before the >> parser even sees them. > > Yes but this was the example: > > X:1 > T:some made up tune > M:4/4 > K:Dminor > abcd|efga|[K:\ > M:3/4 > G]def|gab| > > after dealing with the continued line you get this: > > X:1 > T:some made up tune > M:4/4 > K:Dminor > abcd|efga|[K:M:3/4 > G]def|gab| > > Whether you choose to handle this in the parser or in a preprocessor, > the result still aint legal abc. > > Phil Taylor > > To subscribe/unsubscribe, point your browser to: > http://www.tullochgorm.com/lists.html To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: ABC 1.x continuations (was ABC 2.0 draft)
Except I'm only proposing this as an aid in defining how to parse 1.* style
completions (when the user has selected that as a parsing flag...), not as a
change to the 2.0 spec. I actually prefer the 2.0 continuation mechanism,
although I'm not strongly attached to it and could see how the 1.* style,
defined the way I suggested, could actually make more sense.
(Especially, as I think about it, in light of the %%continueall XCommand,
which certainly should NOT behave as if one had placed a 2.0 style
continuation on each line, but *should* behave, IMHO, as if there was a 1.*
style continuation there...)
As for the "all voices" line, I'm not sure I like that idea all that much --
it creates a new parsing state which opens up a number of cans of worms -
specifically what is legal when you are in an "all voices" section and what
is not.
Possibly a better approach would be using something like [V:*=S] to assign
the current voice specific settings of voice "S" to all other voices. That
field would *not* have the side effect of changing voices. (You might also
allow [V:A=S] or the like.) So your line below:
[V:*] [K:G] [M:3/4]
...might instead read:
[V:S] [K:G] [M:3/4] [V:*=S]
...and you would end up being in voice "S".
I don't know if I like that much either (you still need to define *exactly*
which voice specific settings get copied and which do not), but from a
parsing point of view, I much prefer it over what you were suggesting.
-->Steve Bennett
> I like this way of doing it. I think that it is clearer than
> what I was proposing a couple days ago. This would mean that
> the backslash at the end of a line does not have the semantics
> currently written in the 2.0 spec, but it is more backward
> compatible with 1.x, it is defined well enough to parse without
> ambiguity, and it does everything that seems to be needed.
>
> If we had the ability to specify an "all voices" line, that
> would complete it :-). Then we could write my previous example
> as (with a little more detail):
>
> X:1
> T:test
> M:C
> L:1/8
> K:D
> [V:S] ccAbccA2| fdab\
> w:a-a-a-a a-a-a-a | a-a-a-a \
> [V:A] ccAbccA2| fdab\
> w:i-i-i-i i-i-i-i | i-i-i-i \
> [V:*] [K:G] [M:3/4]
> [V:S] xd | c3 c3 |
> w:a-a | a a |
> [V:A] xd | c3 c3 |
> w:i-i | i i |
>
> And it is perfectly clear what it is supposed to do. There is
> no ambiguity about what the backslashes relate to. Although,
> you should modify your statement so that it can occur on
> tune, lyric, and symbol lines (add the symbol to the list).
>
> tom
>
> Steven Bennett said:
>>>> So... If it can only occur on tune and lyric lines, and it can only
>>>> occur
>>>> where a staff break or lyric line break would be valid, then that is
>>>> why I
>>>> suggested a definition along the lines of:
>>>>
>>>> "A backslash ("\") at the end of a line means do not break the staff or
>>>> lyric line at this point if that's what would happen because of the
>>>> following line ending."
>>>>
>>>> It becomes pretty straightforward (actually fairly easy) to parse if
>>>> you
>>>> define it that way, and seems fairly consistent with the 1.* usage.
>>>>
>>>>
>>>> Anyone see any gaping holes in that logic?
>>>>
>>>
>>> You have still left open the question of whether programs should look
>>> for a simple continuation on the next line, or look for a field
>>> identifier if appropriate. (e.g. a w: field)
>>
>> Actually, I think that by phrasing it the way I did above, there is no
>> assumption made at all about the next line, so it's parsed exactly as you
>> would parse any other field/tune data line. I'm basically avoiding the
>> term
>> "continuation" in 1.* parsing because that confuses the issue -- I see it
>> as
>> not so much a continuation as a break/no-break flag on the line.
>>
>> The assumption I *am* making is that the decision to break the staff or
>> lyric line is made when the end of line is parsed. Without a "\"
>> character,
>> the staff or lyric line is normally broken. With a "\" just before the
>> end
>> of line, the staff or lyric line is not broken.
>>
>> Thus, for a w: field with a "\" at the end, you *would* have the "w:" on
>> the
>> next lyric line, which doesn't have to happen immediately.
>>
>> Of course, this begs another 1.* specific question - should I ignore "\"
>> at
>> the end of non tune data or lyric lines, or flag them as parse errors or
>> warnings? For simplicity, I'm leaning towards simply ignoring them.
>>
>> -->Steve Bennett
>>
>> To subscribe/unsubscribe, point your browser to:
>> http://www.tullochgorm.com/lists.html
>>
>
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: ABC 1.x continuations (was ABC 2.0 draft)
>> So... If it can only occur on tune and lyric lines, and it can only
>> occur
>> where a staff break or lyric line break would be valid, then that is
>> why I
>> suggested a definition along the lines of:
>>
>> "A backslash ("\") at the end of a line means do not break the staff or
>> lyric line at this point if that's what would happen because of the
>> following line ending."
>>
>> It becomes pretty straightforward (actually fairly easy) to parse if
>> you
>> define it that way, and seems fairly consistent with the 1.* usage.
>>
>>
>> Anyone see any gaping holes in that logic?
>>
>
> You have still left open the question of whether programs should look
> for a simple continuation on the next line, or look for a field
> identifier if appropriate. (e.g. a w: field)
Actually, I think that by phrasing it the way I did above, there is no
assumption made at all about the next line, so it's parsed exactly as you
would parse any other field/tune data line. I'm basically avoiding the term
"continuation" in 1.* parsing because that confuses the issue -- I see it as
not so much a continuation as a break/no-break flag on the line.
The assumption I *am* making is that the decision to break the staff or
lyric line is made when the end of line is parsed. Without a "\" character,
the staff or lyric line is normally broken. With a "\" just before the end
of line, the staff or lyric line is not broken.
Thus, for a w: field with a "\" at the end, you *would* have the "w:" on the
next lyric line, which doesn't have to happen immediately.
Of course, this begs another 1.* specific question - should I ignore "\" at
the end of non tune data or lyric lines, or flag them as parse errors or
warnings? For simplicity, I'm leaning towards simply ignoring them.
-->Steve Bennett
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Re: ABC 1.x continuations (was ABC 2.0 draft)
I. Oppenheim wrote:
>> As for the 1.6 and 1.7.6 specifications, regardless of what program X, Y, or
>> Z does, the written spec is awfully vague. I have several possible
>> approaches to different elements of this, but the basic concept appears to
>> be that "\" at the end of a line isn't so much a continuation, but a "don't
>> break the staff here if you would normally".
>
> No. In 1.* it basically meant: continue on the next
> line of the same type. (Whereas in 2.0 it means:
> continue on the next physical line, regardless of the
> context)
I don't think I can accept that definition, even though looking at the
archives, I've seen that definition before. It implies far too many things
which are problematic and not at all true for the programs I have here. Of
course, I've only been testing with Barfly 1.52 and abcm2ps 3.7.18 right
now. Doubtless other programs behave differently.
Problem 1: It implies that you could break a line arbitrarily. For the ABC
2.0 style continuations, that is indeed the case, but we're talking about a
1.* specific parser here, where you can have other lines in-between. So if
you allow breaking a line arbitrarily, you could end up with something like
this:
X:1
T:some made up tune
M:4/4
K:Dminor
abcd|efga|[K:\
M:3/4
G]def|gab|
...which would need to cache part of the first line (the "[K:" part) until
the next tune data line arrived and it had the remaining info it needed to
finish parsing stuff. Even without the meter change in the middle, both
Barfly and abcm2ps give parse errors on that.
And it doesn't need a field -- try something like:
abcd|efga|c'\
''def||
...and you still get errors with both programs, even with nothing inbetween
the lines.
Likewise a lyric line would need to cache partial syllables. Take:
X:2
T:another made up tune
M:4/4
K:Dminor
abcd|efga|bcde||
w:a1 b2 c3 d4 e5 f6 g\
w:7 a8 b9 c10 d11 e12
Now here, Barfly apparently only uses the abc 2.0 style continuations (for
lyric lines - odd that it doesn't do the same for tune data lines...), so it
always ends up with "gw:7" as a syllable, but ends up with the syllables in
the right place. abcm2ps renders this with the g and the 7 as separate
syllables, so all the following syllables are off by one note.
All of which leads me to conclude that under 1.*, you cannot continue a tune
line or a lyric line at any arbitrary character point -- you need to
continue it at a point where a staff or lyric line break would be valid.
Problem 2: It doesn't have any inherent limitation on what "line of the same
type" could be. Which means you could continue just about any sort of line.
Such as:
X:3
T:yet another made up tune
M:4/4
K:Dminor
abcd|efga|[K:G]\
M:3\
bcde|\
M:/4
def|gab|
...complains about the split Meter line in both Barfly and abcm2ps. (With
or without the intervening bcde line...) So it appears to me that it can't
work with just any arbitrary field -- IMHO, the only fields it even makes
sense to use it on (under 1.*) are tune body lines and lyric lines.
So... If it can only occur on tune and lyric lines, and it can only occur
where a staff break or lyric line break would be valid, then that is why I
suggested a definition along the lines of:
"A backslash ("\") at the end of a line means do not break the staff or
lyric line at this point if that's what would happen because of the
following line ending."
It becomes pretty straightforward (actually fairly easy) to parse if you
define it that way, and seems fairly consistent with the 1.* usage.
Anyone see any gaping holes in that logic?
-->Steve Bennett
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Continuation Lines
Hi! I'm in the process of writing Yet Another ABC Parser as part of a larger project I've been working on for a while. (There are umpteen jillion reasons why I'm not using an existing parser - the biggest being this one is written in Objective C, but that's besides the point...) Ideally I'd like this to be able to handle files which conform to ABC 1.6, ABC 1.7.6, and the ABC 2.0 draft spec (dated Aug 14, 2003), but there are numerous little quirks and differences. The quirk I'm wrestling with today is Continuation Lines, which seems to be the biggest single area (and the biggest can of worms on the list archive) where the 1.6 & 1.76 specs seriously conflict with the 2.0 draft spec. Files written one way will parse ONLY if you use the 1.6/1.7.6 method, and files written another way will parse ONLY with the 2.0 method. And I don't see any mechanism (short of a good AI) to figure out which way during the parse. So I'm going to implement both and allow the user to decide which to use via a runtime switch. So... The 2.0 continuation line specification is almost trivial to implement. The only area which could use a bit of clarification there is it's effect during History fields. For example: ... H: This is some history This continues the history So does this And this too %%and this isn't an xcommand % But is this a comment? If you display % the history field, is it included? % I believe the answer should be No... And what about this? X \ Would this appear on the same line as the "X" \ If the history was displayed to the user? (IMHO, Yes) And here's the real challenge: \ % oh boy O: Is this the Origin or part of the History? I think it's part of the history myself And that *this* line is the last line of the history. O: This is the real origin... ... Does this interpretation of the ABC 2.0 variant make sense? Comments aren't actually part of the history field, and continuations *do* apply? As for the 1.6 and 1.7.6 specifications, regardless of what program X, Y, or Z does, the written spec is awfully vague. I have several possible approaches to different elements of this, but the basic concept appears to be that "\" at the end of a line isn't so much a continuation, but a "don't break the staff here if you would normally". Taking that as a given, then "\" would only be meaningful on tune body or lyric lines, and ignored (possibly generating warnings) on other fields. It would not allow putting field-like text inside a History field as described above. You would also require the "w:" at the start of the continued lyric line, unlike 2.0 which would require it NOT be there. Does anyone see anything I'm missing here? The question becomes how to deal with comments. I saw plenty of discussion of whether the "\" is ignored if it follows a comment, or whether it's invalid if it isn't the very last character on the line. I guess it depends on whether you look for the "\" first or the comment first. Is there any consensus out there as to which is the proper approach when you are parsing files using pre-2.0 continuations? (Or should I make *that* a user switch as well...) -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] ABC Standard 2.0 revision III
Bert Van Vreckem wrote: > Bernard Hill wrote: >> In message <[EMAIL PROTECTED]>, Bert Van Vreckem >> <[EMAIL PROTECTED]> writes >>> Bernard Hill wrote: >>> 2. What's a "roll" (+roll+ in the decorations)? I've checked 6 music dictionaries and books on notation and the only rolls mentioned are for timpani or other percussion and notated as either "tr" or a tremolo. >>> >>> It is used at least in Irish music as a general ornamentation mark. I've >>> come across the notation a.o. in "Traditional Irish Music: Karen Tweed's >>> Irish Choice," Dave Mallinson Publications, 1994. >> >> Thanks. But what does it mean? What would say an autoharp make of it, >> say perhaps to make it a tremolo. > > It means "play any ornamentation here". The exact meaning is unspecified. True, more or less -- but then you could say in Irish music that *all* ornamentation is optional and could be played any way you like -- I know some Irish musicians who rarely if ever play the same tune the same way twice. But at least semi-officially, a "roll" in the Irish and Scottish tradition is one of several mechanisms used by fiddle, flute, tinwhistle, and pipes players to handle repeated identical notes without requiring a break in the sound. Fiddle players use it to do the notes in one long bow stroke, flute and tin whistle players use it for playing them in one breath without tonguing, and pipers (bag- & uilean) use it because they need *something* to interrupt the repeated notes -- you don't stop a pipe from making noise easily. And other instrument players don't need it for any reason, but it sounds good. (I use rolls frequently when playing Irish music on my Hammered Dulcimer, for example...) The roll basically inserts higher and lower notes in-between the identical notes. What it works out to is this: Say a tune has a point somewhere that has three A notes in a row. You might write this down as a single 3 beat A note with a roll ornament. When you play it, you will actually play 5 slightly shorter notes in the same timeframe, most frequently ABAGA, but I've seen people play it AGABA or even some other combination -- it's a very loose convention. And there can be longer rolls, but the basic concept is you are rolling up and down around the specified note. That all said, I don't think I've ever actually *seen* any Irish music with a roll ornament actually placed (didn't even know there was a symbol for it until I read this thread...) -- as I said before, Irish players prefer to ornament as they see fit, so the idea of actually writing an ornament on the music seems like you're telling them they can't do that... Of course, most of my written Irish music is tin whistle oriented -- maybe that symbol is used more in fiddle or pipe music. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Unicode and UTF8?
Has anyone given any thought to supporting Unicode and/or UTF8 files in the new standard? Much of my text editing nowadays (I work on Mac OS X...) produces Unicode encoded text files by default, and I think this is going to become more commonplace over the next few years. Also, supporting Unicode would open ABC up to more foreign languages. To accomplish this in a parser, I would actually convert the file to Unicode (if not already) at the very beginning and parse the whole thing in Unicode. One minor change to the spec would be how we deal with special character and continuation processing with the backslash -- I'd do that immediately after everything is converted to Unicode, before any other processing of the file. This results in some subtle changes in interpretation, which means a line like: \101: Steve Bennett ...would now be interpreted the same as: A: Steve Bennett ...which I think is a behavioral change for existing programs, but shouldn't have much of a side effect otherwise. (I'm not sure how to deal with the "\-", which inserts a hard hyphen in a W: or w: field -- maybe it's the only thing not translated, or maybe it gets converted to a Unicode 00AD, which is a Soft Hyphen, although that's kind of reversed in meaning. Or maybe Unicode 2011, which is a Non-Breaking Hyphen, but again that's not *quite* the meaning...) I also suggest adding a Unicode escape to allow just about any Unicode character in an ASCII encoded file. Something like: \U262F ...where "262F" is the hexadecimal value for the Unicode character to insert, in this case a Peace symbol. Of course, an ABC file already in Unicode form could allow any Unicode character to be included in any text field, such as the Word fields, etc. Any comments? -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: %%ABC 2.0 identifier
Phil Taylor wrote: >> (IMHO, >> there should be NO data in an ABC2 file which exists outside the context of >> the current tune -- I don't know if anyone else agrees with this, but it >> makes the most sense to me...) > > I have to disagree. One very useful feature of abc is the possibility > of embedding abc tunes in other text. This makes it a fine tool for > writing about music. See Jack Campin's tutorial on modes in Scottish > music for a good example of this. Oops. I see I really phrased that one wrong. I've put ABC tunes embedded inside emails myself. What I meant to say is that there should be no ABC2 data outside the context of an individual tune inside a file. Each ABC2 tune should be capable of being parsed outside the context of the file it's in. Unfortunately, doing so means more or less doing away with the file header support, or at least disallowing any field that's affects how the tune is processed. (ie. M:, m:, R:, T:, U:) But I expect there would be quite a bit of resistance to that idea, which is why it's just an opinion, and not something I was suggesting we actually do. I *do* think the %%ABC2 tag on each compliant tune, or something similar, is needed. And given we have the file header anyway, I could accept a compromise that the %%ABC2 tag could be placed in the header and thus apply to all tunes in the file. The header has to be copied out anyway whenever you break a tune out of a file, so that's not a big deal. >> (Which reminds me -- is >> there ANY other open source parser out there other than the several dozen >> variants of the ones in ABC2PS and ABCM2PS? I'd like to look at one, >> especially an object oriented one if one exists...) > > There's the parser used in abc2midi and yaps, but it's ansi C not C++. Thanks, I'll look at them. C++ isn't necessary - I used to be a big C++ advocate but now I *much* prefer Objective-C. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: %%ABC 2.0 identifier
Phil Taylor wrote: > To return to the original suggestion in the title of this thread, it > has been suggested many, many times before. One of the first postings > I ever made to this list (or was it the developer's list?), made > exactly the same suggestion - for goodness sake lets write the version > of abc used in this file at the start so programs know what they have > to deal with. It's a great idea, but it won't work because: > > (1) The fundamental unit of abc data is not the file, it's the tune. > ... [snip] > So the identifier would have to be added to individual tunes to be > useful, rather than go in the file header. I fully agree with you. My suggestion is exactly that - this belongs in the header of each tune, and in fact, might be best defined as the first line of a new tune, making it a much more easily identifiable tune break. (IMHO, there should be NO data in an ABC2 file which exists outside the context of the current tune -- I don't know if anyone else agrees with this, but it makes the most sense to me...) > (2) A surprisingly large proportion of users type their abc directly > in a text editor. There's no way you can make these people include > the identifier. They mostly won't bother, and if their software enforces > it they'll prefer to use a program which doesn't. Which is why I suggest it as part of a new ABC 2.0 standard. If *all* programs which support ABC 2.0 need that line, then people will use it when writing new tunes to that format. It's not like typing "%%ABC2" at the start of your tune is a particularly onerous or time consuming task. > If you really want to write an abc parser, you will have to get your > head round the idea that abc is not a computer language, it's a > natural language (albeit a very simple and regular one, with very > few dialects). It makes writing a parser a much more interesting > task. If you want an easy (to the point of being boring) job, write > a MusicXML parser. The issue with XML is you end up with something trivial to parse, but nearly impossible to write without machine assistance. The main reason I like ABC (and I suspect others do also) is it's relatively easy to type in music in that format. Unfortunately, ABC as it exists now has indeed become a natural language, and that limits it's usefulness immensely in the computer world. And I beg to differ about "few" dialects -- as near as I can tell, there are at least as many dialects as there are programs to handle ABC, and from comments here on this list, I suspect there are even more than that. I have no problem developing a complex parser -- I've done that plenty of times before, but developing one from scratch which has to deal with multiple conflicting rules and absolutely no means of identifying which rule applies, is absolutely no fun at all. Writing a parser isn't the goal - it's a necessary step in creating a new program, and there's no reason it should be as bloody difficult as it currently is. (Which reminds me -- is there ANY other open source parser out there other than the several dozen variants of the ones in ABC2PS and ABCM2PS? I'd like to look at one, especially an object oriented one if one exists...) Maybe I should be proposing that the new standard not be called "ABC 2.0" at all, but something entirely different. It would still be the same goal -- establishing a truly standard computer-readable dialect of ABC, but by changing the name, it loses the baggage that name currently carries... -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] Re: %%ABC 2.0 identifier
On Fri, 18 Jul 2003, Jack Campin wrote: [mostly snipped because it was such colorful rhetoric...] > ...this is such ... Authoritarian Actually, I have to admit it *is* rather authoritarian of me. 20+ years of dealing with incompatible and poorly defined computer file formats, protocols, and APIs tell me that when developing a *standard*, the standard writers MUST be authoritarian about it. Strictly so. Or it will cause problems down the line. ABC is *both* a musical notation format, and a computer standard. At least according to the docs, and the history of this thing, it developed as a means of writing tunes in a format that could be understood and printed by a computer, and that appears to be it's primary purpose. If you want to write something that's readable primarily by humans, there are already plenty of non-computer notations to do it in. Unfortunately, ABC right now suffers because nobody *has* taken that strict authoritarian stance -- everyone adds their own little pet additions in 50 zillion incompatible ways, with the result that developing a new parser from scratch (something I wanted to do...) is nearly impossible because of all the variations. Either someone steps in, takes charge, and develops a new, more restrictive standard, or the format keeps bifurcating into something more or less useless. It looks like most of the people on this list are trying to take that step -- I just wanted to add in something that I've found key in dozens of other similar cases I've encountered in my career. Also you seem to be assuming that by creating a strict standard, you lose any option of adding something special that you need but that isn't covered by that standard. Nonsense. User defined fields are commonplace in most standards -- there's nothing preventing us from defining something similar, and I encourage as much. The key is to define user fields in such a way that they can be safely ignored by computer programs trying to parse the file -- that hasn't always been the case with the old ABC, and seems to me to be the driving reason behind developing an ABC 2.0 standard in the first place. And if you could care less if your ABC files are parsable by any computer program, well, feel free to put in whatever you like. I'm not stopping you. The programs will probably complain, toss out the tune, or even crash when your so called "ABC" files were read, but that's no different than what exists now, right? >> If that isn't a strong enough incentive, then it might be a good idea >> to create a program which converts ABC tunes to ABC2 tunes, and then >> don't even bother including a parser for pre-ABC2 files in new ABC2 >> compliant tools. > > And the burn-the-past attitude of that paragraph is obscene anti- > intellectual garbage. You do NOT alter somebody else's past work > without their permission or misrepresent your own editing of it as > being the real thing. Okay, this one I didn't snip completely, because it *really* annoyed me -- it was arrogant and totally uncalled for. Okay, so was most of the rest of the message, but this implies I steal IP, and that's something I feel very strongly about. Read what I said, willya? I did NOT say, or even imply, that we're going to go willy-nilly converting all the tunes on the web to ABC2 and upload the new versions as a replacement -- I don't have the authors permission, as you so very rightly pointed out. Nor would I consider running everyone's ABC files through ABC2PS and upload the resulting postscript files for the same reason. The presence of a tool to convert ABC to a usable ABC2 file is no different. The converted tune doesn't have to be kept around after. I was thinking very much along the lines of a UNIX filter -- ie. "abc2abc2 old.abc | abc22pdf" takes your old ABC file converts it to ABC2 format, and from there to PDF using a new ABC2 to PDF conversion program. Oh, and before you complain about *that*, I wouldn't upload the PDF files, either. Sheesh. Note that with such a program, there is no need to cleanup old ABC files -- any time you want to use an old ABC file with your new program, you simply run it through the converter. The whole idea of this isn't converting old ABC files, it's making it easier to conform to a standard for NEW ABC files. I honestly don't think it would be necessary or worthwhile to try to convert and replace all the existing ABC files on the web once a tool like this exists. And it also significantly lowers the bar for entry for people developing new ABC programs -- you don't have to support the idiosyncrasies that existed before the new standard. You could if you really wanted to -- in fact it would be easy -- just take the abc2abc2 source and stick it in your program as a preprocessor before your ABC2 parser. But you don't have to. -->Steve Bennett To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Re: %%ABC 2.0 identifier
The whole idea (and the goal any standards maker should be working towards) is to provide a mechanism and incentive for both the content developer and the tools developer to implement and stick to the new standard. This is useful to both -- the tools developer should use an old parser if the version tag is not there, and can develop a clean and robust parser for those tunes which have it. The incentive for the content developer to put these in is twofold -- First, if they stick to the new standard, their content should work with ANY ABC2 compliant app. Second, since the old parser isn't being further developed (and shouldn't be, IMHO...), if they want to use any ABC 2.0 or later feature or different syntax, they *must* put in the version tag and comply with that spec, or that feature or different syntax won't be recognized. It's important to establish that this tag isn't a comment or a suggestion to the user or tool -- it's a clear, unequivocal statement: "The following tune conforms *strictly* to the ABC 2.0 spec, and if you find it doesn't, throw the tune out with an error". And it's absence is another clear statement: "The following tune is pre-2.0, so try to parse it as best as you can, and definitely do NOT allow any 2.0 specific features". If that isn't a strong enough incentive, then it might be a good idea to create a program which converts ABC tunes to ABC2 tunes, and then don't even bother including a parser for pre-ABC2 files in new ABC2 compliant tools. The added step of doing the conversion every time they want to try their new tunes out will quickly convince content writers to use the %%ABC2 tag. And it will make writing new tools easier because they won't have to deal with any parsing oddities. Again, IMHO, -->Steve Bennett >> it's fairly clear to me that the real key to >> writing any new specification is adding some identifier that says >> that the tune conforms to the new specification. For example, maybe >> something like: >> >> %%ABC3.0 >> >> ...anywhere in a tune's header will indicate that the tune's encoding >> conforms *strictly* to the ABC version 3.0 (just an example version...) >> specification > > yes, it has already been requested before. As an ABC user, I > totally agree with the idea, and I would include such a comment in > all my tunes (that conform to it) if it enters in the standard. > But the general answer about such an identifier is that "no one > would bother adding it", "you can implement it but only 5 % users > would use it" etc. > > I don't see were is the problem. If some pple don't mind their > tune won't be well played / displayed in most advanced > application, and if they don't wish to add the level of abc used > 1.6, 1.7 or 2.0, it'd be only their own fault. > >> If it's present, though, you can parse it strictly to the >> 3.0 specification -- no deviations allowed. > > it will allow to give responsibilities to pple who wish it this > way, so I find it positive and more logical. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
[abcusers] Re: abcusers-digest V1 #939
Hi! I've been following the list for only a short while now (joined it because I'm considering writing a completely new ABC program using Cocoa/GnuStep/Objective C), and have been following the discussion of how to approach a new version of the standard with interest. I don't know if what I'm about to suggest has been suggested before, but it seems like a fairly critical idea if the community is serious about developing a new standard. Coming from a background of having written a variety computer communications protocols and APIs, it's fairly clear to me that the real key to writing any new specification is adding some identifier that says that the tune conforms to the new specification. For example, maybe something like: %%ABC3.0 ...anywhere in a tune's header will indicate that the tune's encoding conforms *strictly* to the ABC version 3.0 (just an example version...) specification. (Note -- I rather agree with some of the comments here that there should be no file level commands in the next spec - everything should be internal to the tune...) If the version identifier is missing, you parse the tune as you would any previous ABC tune and deal with the idiosyncrasies of the old format however you like. If it's present, though, you can parse it strictly to the 3.0 specification -- no deviations allowed. If you know you only recognize ABC version 2.9 and earlier, you can *try* parsing it to that spec, but should warn the user why it might not work. Once you have that versioning mechanism in place, you can revise the ABC spec to something *far* more internally consistent, and deprecate items from the spec that are problematic. You could even make changes to the spec that would break older non-conforming programs if you like. (Might not be a bad idea to do just that as a means of nudging users of those older non-conforming programs to upgrade...) Finally, once the new standard is hammered out, it *has* to be a strictly conforming standard -- parsers can't be allowed to ignore deviations. Otherwise, program XYZZY will come along and add something unexpected, it will fork into MXYZZY and YZZYX, and before you know it you're back where you started. Some kind of RFC mechanism can be used to propose, comment on, review, and implement changes. IMHO, -->Steve Bennett Phil Taylor wrote: > Yeah, that's the rub. The thing to bear in mind is that we don't have to > stay compatible with abc2win, since the people who use that program don't > need the more advanced features which we are adding to the language. > On the other hand, we do have to maintain some compatibility with abc2win's > files, because they are priceless. What we must avoid is adding something > to the language which prevents new programs which conform to the newest > standard from being capable of dealing with abc2win's files. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
