I had to build something for this
<https://github.com/jackfirth/resyntax/blob/master/syntax-replacement.rkt>
in my Resyntax project. My takeaways were:
- There's no substitute for just reading the file. If you have a
`syntax-original?` subform, you can use the srcloc information to read the
exact original text that was in the source file. This not only preserves
whitespace more accurately than `syntax->string`, it also preserves
comments, which is something that `syntax->string` fundamentally cannot do.
- For non-original syntax, such as that synthetic "comments" bit you're
inserting programmatically, just ignore the source locations completely.
They refer to the source location of where it occurs in your
program-manipulating-program, which is completely unrelated to the
positions of the original syntax. The `syntax->string` form doesn't handle
it well when some subforms are original and some aren't, or when subforms
are from different sources.
- It's handy to have a way to communicate explicit formatting. I did
this by having a special NEWLINE form that my code could shove into the
syntax object before rendering it to a string, to tell the formatter where
to insert line breaks.
- If you just rely on `syntax-original?` to preserve
whitespace/comments, you'll miss the whitespace and comments *between*
original syntax objects. I dealt with this by having an `(ORIGINAL-SPLICE
original-syntax ...)` form that I used to tell the formatter that not only
are all of these syntax objects original, but the entire sequence is
unchanged from the input program. That allows the formatter to copy the
source file text from the start of the first syntax object all the way to
the end of the last one, preserving any whitespace and comments in between
them.
On Friday, April 9, 2021 at 4:22:31 PM UTC-7 Jeff Henrikson wrote:
> It turns out that I have more trouble with printing whitespace from
> syntax. Consider this:
>
> (require racket)
> (require syntax/to-string)
>
> (define (syntax-on-lines-v1 xs)
> (define (iter fin ys)
> (let ((y (read-syntax "<string>" fin)))
> (if (eof-object? y)
> (reverse ys)
> (iter fin (cons y ys)))))
> (if (null? xs)
> '()
> (let* ((fout (open-output-string))
> (_ (for* ((x xs))
> (writeln x fout)))
> (_ (close-output-port fout))
> (fin (open-input-string (get-output-string fout))))
> (port-count-lines! fin)
> (iter fin '()))))
>
> The basic case works:
>
> ;; ex1
> (let* ((xs '("collected" "from" "separate" "lines"))
> (ys (syntax-on-lines xs)))
> (syntax->string
> #`(#,@ys)))
> ;; "\"collected\"\n\"from\"\n\"separate\"\n\"lines\""
>
> But I need to decorate a number of constant items, such as:
>
> ;; ex2
> (let* ((xs '("collected" "from" "separate" "lines"))
> (ys (syntax-on-lines xs)))
> (syntax->string
> #`(comment #,@ys)))
> ;; "comment\"collected\"\"from\"\"separate\"\"lines\""
>
> In ex2, the loss of line breaks seems to stem from lack of srcloc info on
> the first token. However, I don't understand just what's necessary to keep
> things moving along. If I populate the first token only with
> quasisyntax/loc as follows, I get my EOLs:
>
> ;; ex3
> (let* ((xs '("collected" "from" "separate" "lines"))
> (ys (syntax-on-lines xs))
> (zs (cons (quasisyntax/loc (car ys) "comments") ys)))
> (syntax->string
> #`(#,@zs)))
> ;; "\"comments\"\"collected\"\n\"from\"\n\"separate\"\n\"lines\""
>
> But that's quite inconvenient if I have a bunch of stuff to decorate. If
> I try to do the more convenient outside position for quasisyntax/loc, my
> EOL data seems to get overwritten:
>
> ;; ex4
> (let* ((xs '("collected" "from" "separate" "lines"))
> (ys (syntax-on-lines xs)))
> (syntax->string (quasisyntax/loc (car ys) ("comments" #,@ys))))
> ;; "\"comments\"\"collected\"\"from\"\"separate\"\"lines\""
>
> Is there a way to do something like ex4, where I can add a number of new
> constant tokens, but without overwriting the EOL data?
>
> Thanks in advance,
>
>
> Jeff Henrikson
>
>
> On 4/9/21 9:57 AM, Jeff Henrikson wrote:
>
> Laurent,
>
> Thank you very much. It probably would have taken me a long time on my
> own to think of the possibility that the port was at fault.
>
>
> Jeff
>
>
> On 4/9/21 4:29 AM, Laurent wrote:
>
> You need to enable line/character counting with `port-count-lines!`:
>
> #lang racket
> (require syntax/to-string)
>
> (define in (open-input-string "(comment\n \"hello world\"\n line)"))
> (port-count-lines! in)
> (syntax->string (read-syntax "mystring" in))
>
> ; -> "comment\n \"hello world\"\n line"
>
> On Fri, Apr 9, 2021 at 2:10 AM Jeff Henrikson <jehen...@gmail.com> wrote:
>
>> Racket users,
>>
>> I’m trying to read a scheme file, decorate a bit of window dressing
>> around the edges, and write the expressions to a new file. I’ve got the
>> basic mechanism working with (read . . .) and (pretty-write . . .), but of
>> course that doesn’t preserve linebreaks. So now I’m trying to improve it
>> to preserve linebreaks. It would be nice to preserve all whitespace, but
>> I'll settle for linebreaks. The racket docs seem to suggest it is possible:
>>
>> 2.6 Rendering Syntax Objects with Formatting
>> (require syntax/to-string) package: base
>> procedure
>> (syntax->string stx-list) → string?
>> stx-list : (and/c syntax? stx-list?)
>> Builds a string with newlines and indenting according to the source
>> locations in stx-list; the outer pair of parens are not rendered from
>> stx-list.
>>
>> However, when I evaluate:
>>
>> (syntax->string (read-syntax "mystring" (open-input-string "(comment\n
>> \"hello world\"\n line)")))
>>
>> I get:
>>
>> "comment\"hello world\"line"
>>
>> which has no whitespace at all, not even the whitespace that is necessary
>> to separate the original tokens.
>>
>> I get a similar behavior if I read-syntax from a file and apply
>> syntax->string to those values.
>>
>> Does anyone know how to get syntax->string to recover the original
>> whitespace?
>>
>> I'm using Racket 8.0 cs on Ubuntu 20.
>>
>>
>> Thanks in advance,
>>
>>
>> Jeff Henrikson
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Racket Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to racket-users...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/racket-users/95dd99f5-2d04-a607-ed11-bc235095eeb7%40gmail.com
>>
>> <https://groups.google.com/d/msgid/racket-users/95dd99f5-2d04-a607-ed11-bc235095eeb7%40gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/racket-users/3079ce8d-b051-4000-b120-df8ac246e67bn%40googlegroups.com.
