Maybe we can do a little hackathon on the new parser in a few weeks right after holidays. Things are a little busy for me right now.
Either in person in the Bay Area or remotely with Google Hangouts or something. Best, John. > On Nov 30, 2016, at 5:21 AM, Jon Harper <jon.harpe...@gmail.com> wrote: > > Hi, > just heard today of tagged template literals in ES6: > https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Template_literals > > tag`foo ${bar}` > > which seem to work like what Doug had in mind. That's nice. > > I want to find some time to help with the new parser, which solves real > problems (universal comments, inconsistencies between parsing words, and many > more..) and adds a lot of value to factor ! > Cheers, > > > > Jon > >> On Wed, Aug 24, 2016 at 8:15 PM, Dave Carlton <da...@polymicro.net> wrote: >> Way back long time ago, in the 80's, the old Apple MPW used option key to >> make delimiters easier. Like ∂ as escape, and ¬ for line continuation. Here >> we are, 30 years later still using essentially 7-bit ASCII for textual >> representation of code. Why not take advantage of all those unused >> characters? Like perhaps « block » or >> … >> block of comment text >> … >> >> --- >> Dave Carlton >> PolyMicro Systems >> da...@polymicro.net >> 808-220-1727 >> >>> On Aug 17, 2016, 15:07 -1000, Doug Coleman <doug.cole...@gmail.com>, wrote: >>> (disclaimer: sourceforge only accepts 40kb message bodies without approval, >>> i might have forgotten the admin password, this email might be sent 3+ >>> times, google inbox confuses me with hidden text bloating my replies...) >>> >>> >>> >>> Hi Александр, >>> >>> I'm finally able to contain enough rage to answer you! ;) >>> >>> --Delimiters-- >>> >>> My definition for delimiter is "character(s) that separate the type of the >>> payload from the payload". >>> >>> Some examples are: >>> >>> url"google.com" >>> url`google.com >>> url( "google.com" ) ! hypothetical call to url word with string as payload >>> >>> The delimiter tokens I'm proposing are [ { ( ` " [[ {{ (( and the >>> corresponding closing tags. >>> >>> >>> --Code bloat-- >>> >>> The minimal executable size is based on what the tree-shaker is able to >>> prune from the program. If there's no locals usage, in theory it should not >>> include any code at all for locals, though this would need to be verified >>> and possibly fixed. >>> >>> As for out of order definitions and circularity, it's hellish right now not >>> having these features. If your vocabularies load correctly without >>> circularity but then circularity is introduced through code changes, right >>> now it will likely reload fine but throw an error upon loading in a fresh >>> image. It's hard to track these down in the fep backtrace as it's just a >>> spew of badly-formatted factor objects and separators. Also, the >>> circularity can be subtle, as it usually isn't as simple as reciprocal >>> usage, it's more likely a chain of dependencies and tracking it down is >>> hard without tools (which we don't have). >>> >>> >>> --Loading code from other platforms-- >>> >>> Loading Linux code on Windows etc would really just be loading the textual >>> code and possibly running the stack-checker on it. It would use as much >>> memory as the text size of the file + the factor slice and syntax objects >>> used to contain that text. Of course you would strip these objects on a >>> deployed image. >>> >>> Loading code for the wrong platform means you can do whatever really, as >>> long as you stop short of calling functions that don't exist on the >>> platform. The advantage you get is being able to use a tool to rename a >>> word completely across every Factor file, not just the loaded files or the >>> files that run on your current platform. There are so many bugs in the git >>> history where we updated one platform but not another because the tools are >>> missing. >>> >>> >>> --Arbitrary Payloads-- >>> >>> An arbitrary payload for text is the ability to put any text at all in the >>> string literal without having to escape it. For instance, the common >>> problem in C/C++ is if you have a comment /* ... */ and then you decide to >>> comment out a larger block that contains that comment. You can use the >>> ``#if 0 ... #endif`` trick, but what if you want to comment that block out? >>> The payload (the comment) starts to interfere with the delimiters. Lua has >>> a cool way to contain any text in a literal without escaping it where you >>> just make sure the delimiters are variable and are not contained in the >>> payload. >>> >>> If C comments worked different: >>> /* */ # first C comment >>> /** /* */ **/ # nested C comment, yes this isn't really C >>> /*** /** /* */ **/ ***/ # etc, making sure ******/ isn't in the payload >>> >>> The same principle applies for string payloads. A common use case is >>> copying text off a website or out of a hexdump, out of a packet capture, >>> etc, and not needing to care about escaping the copied text. You can even >>> generate the right delimiters with a tool or the editor if you have the >>> payload. The motivation for this feature is to allow the programmer to >>> forget about string/comment escaping. >>> >>> Finally, features like python's triple strings '''string''' and >>> """string""" (single and double quotes, tripled) are usually ok, but if you >>> want to write docs about them that contain syntax examples, then you have >>> to micromanage your string delimiters. It's frustrating and programming >>> should not be frustrating in this way! >>> >>> >>> --Backticks-- >>> >>> I wanted a way to golf the C string syntax for strings that don't have >>> spaces. The proposed way is url`google.com, where url is the tag and >>> google.com is the payload. Also I thought that you had to double or >>> triple-quote markdown with backticks, but this doesn't appear to be the >>> case, it supports `thing`. >>> >>> C++ has something like this with their user-defined literals: >>> Kilograms w = 200.5_lb + 100.1_kg; // C++ user-defined literals >>> >>> Notice that you don't have to use two escape characters, just the one >>> works. I'm open to not having the foo`bar form and just making it foo`bar`, >>> but that was the motivation -- golfing it one character! Could also use >>> single-quote or abandon the idea. >>> >>> >>> --Delimiter Location-- >>> >>> The <XML XML> vs XML< XML> is a style choice. It's consistent with the >>> "tagged payloads" style where you have a 1) >>> tag-delimiter-payload-[delimiter]. The other way to do it is by 2) >>> delimiter-tag-payload-delimiter, e.g. ``[fry _ + ]`` or ``{H { 1 2 } }``. >>> The 2) way is consistent with the <XML XML> syntax. It's whatever, but the >>> attempt was at consistency. >>> >>> >>> --Concatenative lexing tokens-- >>> … >>> V{ 1 2 3 }[ 0 ] ! should desugar to C-style array access >>> >>> The idea behind the above syntax was that if several lexed tokens are >>> together without whitespace, then the first one decides how to handle the >>> following ones. So a vector followed by a quotation would attempt to >>> address into itself with the return values from the quotation. This might >>> work better with delimiter style 2) from above, like ``{V 1 2 3 }[nth 0 >>> ]``. On the other hand, this looks "ugly"? >>> >>> What do you think about V{ 1 2 3 } vs {V 1 2 3 } ? >>> >>> >>> --Comments-- >>> >>> author# erg ! a "typed/tagged comment" >>> >>> This is indeed a weird idea. There's nothing preventing it from working, >>> but it's probably too confusing. It works better in style 2) perhaps: >>> #author erg >>> >>> >>> --Backtick example-- >>> >>> - How will this case be handled? >>> - fixnum``hello```world````` >>> >>> This one is a mess. According to the lexing rules, it would see fixnum as a >>> tag of a double-backticked payload, concatenated with a single-backtick >>> "world" with no tag, with an empty double-backtick payload with no tag. I >>> dunno. It seems to lex, but the next phase would pattern-match against >>> fixnum + the rest of the mess and fixnum wouldn't know how to handle it, so >>> it would have a parse error. Something like that. >>> >>> Or the trailing backticks would say "end of file found but expected >>> 4-backticks". >>> >>> >>> --Delimiters revisited-- >>> >>> Literals have to start with opening delimiters. Things like ))foo(( should >>> not work. >>> >>> For ``$description{ "foo" }`` in documentation, the $ just signals docs. >>> It's not really special. Maybe there's a better convention. >>> >>> --Operators-- >>> >>> I had problems trying to figure out the ``char: a`` form, but I think it's >>> just a prefix operator named ``char:``. It should parse as ``char:`` ``a`` >>> and in another pass they should be joined into a single token, >>> operator-char: with a payload "a". Likewise color: hexcolor: pointer: >>> alien: are prefix operators, the pair-rocket H{ 1 => 2 3 => 4 } is an infix >>> operator, and something like ``a++`` could be a postfix operator, e.g. ``1 >>> a++`` where it could increment it at compile-time if it's a literal, or >>> dispatch at run-time otherwise. (Postfix operators are unnecessary?) >>> >>> Prefix and infix operators fix the problem of "related text should parse to >>> a single literal" and the char: prefix operator means that lower-case-colon >>> words don't have to be baked into the lexer. Also, the assignment :> >>> doesn't have to change, it's just a prefix operator! >>> >>> Something like: >>> PREFIX-OPERATOR: \ char: >>> INFIX-OPERATOR: \ => >>> >>> --Fry and make-- >>> >>> It seems that '[ _ , % ] syntax is fine for these, and any other ways to >>> write it, like $[ obj% seq% ] are hard to type into the editor and look >>> weird. >>> >>> --Roots-- >>> I haven't figured out the best repository/directory structure, but it >>> should handle adding repositories with arbitrary URIs like ``@erg/factor``, >>> handle versions of Factor libraries, etc. >>> >>> --Final thoughts-- >>> >>> The parser should give better error messages with enough work and syntax >>> error examples. There's really no reason for it to be worse. >>> >>> Sorry for the mishmash of replies. >>> >>> Cheers, >>> Doug >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Factor-talk mailing list >>> Factor-talk@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/factor-talk >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Factor-talk mailing list >> Factor-talk@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/factor-talk >> > > ------------------------------------------------------------------------------ > _______________________________________________ > Factor-talk mailing list > Factor-talk@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/factor-talk
------------------------------------------------------------------------------
_______________________________________________ Factor-talk mailing list Factor-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/factor-talk
