[racket-users] Re: Pitfall for contracts with serializable structs

[Alex Harsanyi]

On Wednesday, February 6, 2019 at 12:35:11 PM UTC+8, Philip McGrath wrote:
>
>
> Why is this significant in practice? Realistically, I don't expect 
> programmers would write functions like `corrupt-serialized` in an attempt 
> to deliberately exploit some library's invariant, and the chance of someone 
> doing so accidentally seems fairly remote. What worries me much more is 
> that I read serializable structs in over the network, stick them in 
> databases, and write them to disk. In the absence of a contract, any struct 
> that has ever been deserialized may be silently corrupt. Some bits could 
> have been flipped or some other process could have mangled a file on disk. 
> An attacker could have intentionally manipulated a value sent over the 
> network as part of some potential exploit. Perhaps most likely, the could 
> have been a backwards-incompatible change (perhaps an unintentional one) to 
> the contract on some field of the struct.
>

> A contract couldn't protect me from encountering a malformed serialized 
> representation, but it would let me detect the error as soon as I try to 
> deserialize the bad value, rather than delaying the error until I actually 
> attempt to access a bad field. As things stand, if I usually just pass the 
> value around and move it in and out of storage, I might be unknowingly 
> passing around bad data indefinitely.
>

Even if you run the contract checks at de-serialization time, the contract 
system cannot protect you from a corrupt packet, as a bit flip might change 
a number into another, so a contract such as `integer?` would still pass,  
or it might corrupt the data such that de-serialization would fail 
altogether.  To protect against network or disk corruption you will need to 
add check-sums to the serialized data, and if you are worried that a 
malicious party might modify the data, you need to add cryptographic 
signatures to the data in the database or use SSL for the network.

... this of course is not a good reason not to check contracts when data is 
de-serialized.
 
Alex. 

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Pitfall for contracts with serializable structs

[Philip McGrath]

I was (relatively) recently bitten by an issue with putting contracts on
serializable structs. What's worse, once I figured out what was going on, I
realized that I'd run into this very issue before and even discussed it on
this list
. In
that case, though, I'd encountered the problem with higher-order contracts
and other complications: I hadn't fully appreciated that the same caveats
apply to much simpler cases.

I want to write up this problem for the list, both because I suspect others
are making the same oversight I was and because I think this problem
deserves a linguistic solution.

In essence, the problem is that, if you are using `serializable-struct`,
any contracts you may put on the struct (usually through a `struct` clause
in `contract-out`) are not enforced when instances are deserialized with
`deserialize`. Here is an example program:

#lang racket

(module server racket
  (require racket/serialize)
  (provide (contract-out
[struct boxed-set ([v set?])]))
  (serializable-struct boxed-set (v)
#:transparent))

(module bad racket
  (require racket/serialize
   (submod ".." server))
  (provide (contract-out
[make-empty-boxed-set/bad
 (-> boxed-set?)]))
  (define (corrupt-serialized s)
(list-set s 6 '(0 "not a set")))
  (define (make-empty-boxed-set/bad)
(deserialize
 (corrupt-serialized
  (serialize (boxed-set (set)))

(module main racket
  (require (submod ".." server)
   (submod ".." bad))
  (boxed-set-v (make-empty-boxed-set/bad)))

The `server` module tries to protect its `boxed-set` datatype with a
contract in just the way I was doing, which I expect many other Racketeers
are doing as well. If I'd written such a module with `struct` instead of
`serializable-struct`, I would know that no other module will be able to
construct a `boxed-set` instance with a bad value. If I know that my own
module doesn't create malformed instances, either (perhaps because it
doesn't construct any instances), I could correctly assume that every
instance satisfies its contract, giving all of the familiar benefits for
all parties when reasoning about their code.

Using `serializable-struct` opens a loophole in this guarantee. The
expansion of `serializable-struct` uses the same lower-level mechanisms
available for implementing a custom serialized representation: it generates
a `prop:serializable` property and a `deserialize-info` value, which is
exported from the `deserialize-info` submodule as
`deserialize-info:boxed-set-v0`. The `deserialize-info` value encapsulates
a function to construct a `boxed-set` instance from the serialized
representation, which is implemented using the `boxed-set` constructor. The
key point is that the expansion of `serializable-struct` uses the version
of the `boxed-set` constructor from inside the `server` module, which *is
not protected by a contract.* Thus, `deserialize` effectively exposes the
contract-less constructor, albeit quite indirectly.

The `bad` module demonstrates how malformed instances of `boxed-set` can be
constructed without triggering a contract violation. The `main` module then
attempts to use `boxed-set-v` on a malformed value it has gotten from
`bad`, triggering this exception, which accurately blames `server` for
failing to live up to its promised contract on `boxed-set-v`:
boxed-set-v: broke its own contract
  promised: set?
  produced: "not a set"
  in: the range of
  (-> boxed-set? set?)
  contract from:
  (<...>/serial-contract.rkt server)
  blaming: (<...>/serial-contract.rkt server)
   (assuming the contract is correct)
  at: <...>/serial-contract.rkt:7.23

The especially pernicious part is that, if `main` hadn't used
`boxed-set-v`, the error might not have been detected until the value had
flowed far away into some unrelated code.

Why is this significant in practice? Realistically, I don't expect
programmers would write functions like `corrupt-serialized` in an attempt
to deliberately exploit some library's invariant, and the chance of someone
doing so accidentally seems fairly remote. What worries me much more is
that I read serializable structs in over the network, stick them in
databases, and write them to disk. In the absence of a contract, any struct
that has ever been deserialized may be silently corrupt. Some bits could
have been flipped or some other process could have mangled a file on disk.
An attacker could have intentionally manipulated a value sent over the
network as part of some potential exploit. Perhaps most likely, the could
have been a backwards-incompatible change (perhaps an unintentional one) to
the contract on some field of the struct.

A contract couldn't protect me from encountering a malformed serialized
representation, but it would let me detect the error as soon as I try to
deserialize the bad value, rather than delaying the error until I actually
attempt to access a bad 

Re: [racket-users] Re: updated Racket-on-Chez status

[Alex Harsanyi]

First, thanks for looking into this.  Rather than answer inline, I will 
just comment a few things:

* Unfortunately, the tests use real data because they try to pick up 
problems with the code, not test the performance, however, some of the 
tests do run against only data from the repository.  I will not make my 
personal training data available publicly, however I gave Matthew Flatt 
access to it and he can run those tests.

* there are two additional tests that can be run directly, they are on the 
"ah/perf-test" branch and are named "test/t-db-insert-activity.rkt" and 
"test/t-df-mean-max.rkt" -- the second one sets up a minimal `df-mean-max` 
test with some real data.  Interestingly, `df-mean-max` runs faster in 
RacketCS in that test, which is not what I found when I run the other tests 
-- this needs more investigation.

* I was aware that using `in-list`, `in-vector`, etc have better 
performance characteristics, but I prefer not to use them unless necessary: 
it is much nicer when code works with a variety of container formats, as 
much as possible, I have occasionally changed data representation, and 
Racket will not pick up that a function which uses `in-list` is now being 
passed a vector, this will result in a run time error instead.  In fact, 
most of the tests just run the code in the hope of triggering contract 
violations.

Which brings me to the last point:  the BAVG set that `get-mean-max-bounds` 
receives will have about 95 items in it, it will be this list [1]. If I 
would use a data-set of 10 million items in BAVG, I would have bigger 
problems: the `spline` function which also uses this BAVG set would run out 
of memory as it would try to construct a matrix of 10e14 items which would 
take 720 terabytes of memory (assuming floating point numbers)

More realistic values are, which I am aiming for:

* input data processing functions, such as `df-mean-max`, should work 
reasonably fast for a data-set of 3600 to 10800 items, which corresponds to 
activities that last 1 to 3 hours.  They should also work acceptably for 
data sets of 21600 items (an Ironman bike split).
* data plot and summary functions should work reasonably fast for data sets 
of 100 to 5000 items (depending on the case) -- in fact my code goes to 
great lengths to ensure that the data that is passed to any plot function 
is reasonably small, so plots are displayed in 1 second or less.

I am always looking for feedback and if you want to spend some time 
diagnosing performance issues, I can setup more individual tests, or I can 
let you know what realistic data sets are so you can setup tests of your 
own.

Best Regards,
Alex.

[1]  
https://github.com/alex-hhh/ActivityLog2/blob/72598c184caf541c4e2a60aa93e19159b10563af/rkt/data-frame/meanmax.rkt#L60

On Tuesday, February 5, 2019 at 9:49:12 PM UTC+8, gustavo wrote:
>
> I have been trying a few variations of the code. It would be nice to have 
> a test branch that use only the data in the repository. I used some fake 
> data instead.
>
> For the tests, I used the function *get-mean-max-bounds* 
> https://github.com/alex-hhh/ActivityLog2/blob/master/rkt/data-frame/meanmax.rkt#L409
>  
> with this data 
>

>   (define fake-data2
> (for/list ([_ (in-range 1000)])
>   (if (< (random) .01)
>  (vector #f #f)
>  (vector (- (random) .5) (- (random) .5)
>
> so, I tested with
>
>   (time (get-mean-max-bounds fake-data2))
>
>
>
> *** The main time improvement was changing 
>   (for ([b bavg] #:when (vector-ref b 1))
> ...)
> to
>   (for ([b (*in-list* bavg)] #:when (vector-ref b 1))
> ...)
>
> This increase the speed to the double or more. In the microbenchmark, the 
> new duration is the 40%-50% of the original duration.
>
> IIUC, in all functions you know the type of sequence of the arguments, so 
> my advice is to add in-list or in-vector to each and every for in the whole 
> file (or project).
>
> This is a good general recommendation. With in-list or in-vector or 
> in-range, the generated code is very efficient. Without them, the code has 
> to create a generic object to track the iteration, and the code is much 
> slower.
>
>
> *** I tried eliminating the set! and using for/fold instead. The problem 
> is that the code is slower :(. In general it's better to avoid mutable 
> variables, but in this case removing them makes the program slower. We 
> should take a look at the internal code of Racket and try to fix it, 
> because in a perfect world the version without set! should be faster. 
> Meanwhile, keep the current version...
>
>
> *** I tried replacing the for and set! with an explicit loop. Something 
> like
>  (let loop ([bavg bavg] [min-x #f] [max-x #f]  [min-y #f] [max-y #f])
>...)
>
> With this change, there is an additional 5% improvement in the speed, but 
> the legibility is reduced too much. So this is better than the version with 
> for and in-list, but I recommend to keep the legible version.
>
>
> *** I tried 

Re: [racket-users] Re: Some concern about ChezScheme...

[Neil Van Dyke]

I appreciate the engineering diligence behind Alex's and Paulo's concerns.

Given the exemplary track record on Racket, I'm comfortable putting 
faith in Matthew's assessments (like a trusted engineering colleague, 
beyond the quantifiable like interim benchmarks), and there's at least 
some obvious goal alignment, and there's also been a good amount of 
transparency and conspicuous methodical work on the Chez work.


Thanks to Alex, I just realized that I don't recall the plans for 
performance at switchover time, and going from there.  It'd be good to 
get an update/reminder on the current thinking, as people have to plan 
for long-term.


(My experience with a Racket production server farm is that we often 
didn't pick up each new Racket version immediately, and one time I 
backported a Racket enhancement across multiple versions when we had to 
be especially conservative, but we wanted to keep the option to move to 
use the latest version at any time, and we liked to know that we're on a 
good track for long-term.)


(BTW, some of us run Racket on ancient Intel Core 2 and older smartphone 
ARM, plus have Racket on a beefy new dedicated real-metal compute 
server, and we use 
"https://www.neilvandyke.org/racket/install-racket-versioned.sh;... so 
we will know if anyone tries any funny-stuff! :)


--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Re: Some concern about ChezScheme...

[Alex Harsanyi]

I guess I also have some concerns about the move to Chez, and largely for 
the same reasons:

* the Chez community is very small, at least when looking at the 
chez-scheme Google Group and Github activity.  I am glad that I'm not the 
only one who noticed that.

* the "maintainability" angle is questionable.  I read all the reports and 
watched the presentation on YouTube, but all I found can be summarized as  
"Matthews opinion is that it will be more maintainable", while his opinion 
carries a lot of weight, it remains an opinion. Time will tell if 
Racket-on-Chez will actually be more maintainable.  BTW, in my professional 
career I have seen a few large ode re-writes whose only benefit was to be 
"improved maintainability", without a clear definition of what that means. 
After the company spending a lot of time and money, this maintainability 
improvement did not materialize...  based on that experience, I remain 
cautious -- sorry, I have been burnt to many times :-) 

* performance wise, it is clear to me by now, that the best I can hope for 
is for overall performance to remain largely the same when Racket 8.0? is 
released.  In the current snapshot, the overall performance is worse.  
There is a separate google-groups thread about this, and I will try to help 
out on this, mostly by providing test cases.

Based on what I have read and seen, my best understanding of Racket-on-Chez 
is that Racket as a language research platform, will probably benefit from 
this move and  Racket as a teaching platform will not be affected either 
way.  What that means for software development using Racket, remains to be 
seen.

I apologize for the negative message...
Alex.

On Tuesday, February 5, 2019 at 9:01:20 PM UTC+8, Paulo Matos wrote:
>
> Hi all, 
>
> Now that I got your attention... :) 
> Although the title is not purely click-bait, it is motivated by personal 
> requirements. 
>
> Most of us are happy with the move to Chez (actually haven't heard 
> anyone opposing it), but I would like to point to something I have felt 
> over the past year and to understand if this is just my feeling or not 
> from others (maybe Matthew) who have had any experience working with Chez. 
>
> I have been, for over a year on and off, trying to port Chez to RISC-V. 
> My problem here is really understanding the Chez arquitecture, rather 
> than the RISC-V one with which I have been working for a few years now 
> as a consultant. 
>
> The whole point of the work was not to get Chez on RISC-V per se but to 
> get Racket on RISC-V. My initial email to the Chez ML was replied to by 
> Andy Keep. He replied in great detail on how to create a Chez backend - 
> I have cleaned up his reply and have been slowly adding to it [1]. 
>
> That was a great start but from there things started to fall apart. 
> Further emails to my questions were generally not replied to (of the 4 
> messages sent, 1 was replied to) [2]. 
>
> Then there is some backend rot... I noticed for example, that there was 
> no Makefile for a threaded version of arm32 although the backend file is 
> there meaning it should be supported. It seems that it's just that 
> nobody every tried to build it. Then software floating point is an 
> option in a backend config file but if you enable it, bootstrapping 
> doesn't work because the compiler really expects you to have some 
> floating point registers. 
>
> Matthew mentions the move to Chez will help maintainability and I am 
> sure he's right because he has been working with Racket for a long time 
> but my experience comes from looking at backend files. When you look at 
> them you end up being forced to look elsewhere, specifically the 
> cpnanopass.ss file [3]. Well, this file is the stuff of nightmares... 
> It's over 16000 (sixteen thousand!!!) lines of dense scheme code, whose 
> comments are not necessarily Chez-Beginner friendly (maybe Alexis wants 
> to rewrite it? [4]). 
>
> So I am a bit concerned about this. I somehow get the feeling that 
> what's going to happen is that Chez is going to slowly degenerate to a 
> Racket sub-project, and nobody is going to really use Chez directly. 
> Therefore this means Matthew et al. will end up maintaining it along 
> with Racket itself. As far as I understand it, both A. Keep and R. 
> Dybvig are both at Cisco and Chez is a side-project from which they are 
> slowly distancing themselves. Chez becoming a sub-project of Racket 
> might seem far-fetched until you noticed Matthew is already the number 4 
> contributor of Chez since it was open-sourced [5]. 
>
> The only question I have is really, what do other people feel about 
> this. Am I making any sense? Have I missed some hidden Chez community 
> that's working day and night into improving it? Or is Chez current sole 
> purpose of existence to support Racket? 
>
> [1] 
> https://github.com/LinkiTools/ChezScheme-RISCV/blob/wip-riscv/PORTING.md 
> [2] 
>
> 

Re: [racket-users] Some concern about ChezScheme...

[Neil Van Dyke]

I had a related (but different and small) concern about the new 
dependency a while ago (this was off-list), but it sounded like that 
risk was covered, and also that Matthew has really gotten into the Chez 
code.


BTW, sometime around when the move to Chez settles, it would be good if 
many people were somewhat familiar with current Racket internals.  
(Personally, I know a little of it, from writing C extensions, and from 
occasional intense optimizing, but I should take a fresh look, and the 
Chez move seems like a great time.)


BTW, students, especially: if you've not yet been on a few projects in 
which you've looked at diffs for every single change (maybe by having 
them emailed), it's a learning experience, giving you a better feel for 
the system and its evolution and the process, and Racket might be a good 
one for that.  Discussions around changes, too, if you have access to 
those.  I might soon do this myself, for Racket and/or Chez upstream, 
although it feels suspiciously like work. :)


P.S., Yay, RISC-V! :)

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Real-world examples of XML usage?

[Neil Van Dyke]

If you're doing JSON that's not totally ephemeral (not like sending a 
few keystrokes for a live search JS widget is [1]), but something more 
like returning information from a database, you could do JSON yet get a 
little closer to some of the benefit of XML (like "what the heck is this 
data that we found in a file, or as an attachment in our database") by 
including a little metadata in the top level of the JSON.


Doing this metadata might also comfort engineers who care deeply about 
systems, [2] and who are always reasoning about uncertainties like 
possibilities and risks.


Such metadata can include a URL for the schema/documentation of the 
message (which references different spec for the metadata envelope), 
what software produced it, a timestamp, maybe what server produced it, 
maybe a more-specific characterization of the content separate from the 
schema but perhaps not implicit in the content (maybe request REST data, 
or request URL), maybe there's an industry sector standard to reference 
(much like some XML DTDs/schemas), maybe you have a data sensitivity 
model or authorization model that's appropriate for tagging sensitivity 
of messages, etc.


If you're working on a startup and moving fast, and there's no standards 
yet, and you don't want to figure out metadata right now, you might just 
decide to use a `write-foo-json-message` procedure everywhere, which 
only leaves a placeholder for metadata in the JSON structure.  When you 
later can figure out metadata, that Racket procedure can get some new 
required keyword arguments for necessary info (giving you compiler 
errors until you visit each call site), though you'll still have to 
search through the JS code, to see where you should be using the 
metadata (but at least the more fragile JS code will keep working, 
because you had the placeholder in the JSON there from the start).


[1] That might be a bad example of ephemeral today, with some of the 
dotcom snoopers capturing and analyzing even fine-grained mouse move 
events. :)


[2] I care, without being married to anything, but it's unsettling how 
interesting mundanity of XML and JSON details can be, for a moment. :)


--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Real-world examples of XML usage?

[David Storrs]

On Tue, Feb 5, 2019 at 11:18 AM Konrad Hinsen
 wrote:
>
> David Storrs  writes:
>
> > to type things.  In addition, most developers that I've worked /
> > talked with will typically reach for the JSON API before the XML one
> > given the choice.  I think the ground truth suggests that JSON is a
>
> Ah, I see, we are working in very different contexts. In a Web API, I'd
> go for JSON as well. After all, you need little more than a
> serialization format for the input and output parameters of the API functions.
>
> But my main use case is data storage in self-contained files stored in
> databases (including Git repositories), to be read by existing or
> yet-to-be-written software. The data can be quite complex, so any
> machine-readable description of its structures is a plus. XML on its own
> already provides the tag structure, and a schema adds much more
> documentation, still in a standard format.

Ah, I see.  So the answer to "What is a real-world case where it makes
sense to use XML over anything else?" is "When storing data on
disk."[1]  Okay, I can get behind that.  Thanks.


[1] DBs are a special case of 'on disk'.
>
> Konrad.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Some concern about ChezScheme...

[Matthew Flatt]

Hi Paulo,

Not to discourage other answers to your call for opinions, but here's
mine.

Granting your point about the structure of the code in Chez Scheme,
everything is relative. I still think Chez Scheme is a better starting
point than the existing Racket implementation as code to reorganize,
document, and improve. Adding a comment or two in the source is likely
a part of that process. :) We're not at a point where it makes sense to
recommend big changes to the organization of the Chez Scheme source,
but that could happen.

I think you're mistaken about Andy and Kent's position on Chez Scheme.
They will certainly not distance themselves from the project any more
than I could conceivably distance myself from Racket. But they have
priorities besides improving something that is probably just about
perfect for their purposes.

We're under no illusion that moving to Chez Scheme will change much
about who contributes to the core Racket implementation. Based on past
experience (e.g, moving `racket/gui` from C++ to Racket), the
contributor rolls will grow, but not radically. I'm delighted that you
have been willing try a RISC-V port, and even the attempt is a kind of
success and will help things improve things through feedback like this.

Personally, while my contributions to Chez Scheme so far have been
modest, I have already factored into my costs the worst-case scenario
of fully maintaining Chez Scheme as used by Racket. Even if that
happens, it still looks like a good deal in the long run.

Matthew

At Tue, 5 Feb 2019 14:01:07 +0100, "'Paulo Matos' via Racket Users" wrote:
> Hi all,
> 
> Now that I got your attention... :)
> Although the title is not purely click-bait, it is motivated by personal
> requirements.
> 
> Most of us are happy with the move to Chez (actually haven't heard
> anyone opposing it), but I would like to point to something I have felt
> over the past year and to understand if this is just my feeling or not
> from others (maybe Matthew) who have had any experience working with Chez.
> 
> I have been, for over a year on and off, trying to port Chez to RISC-V.
> My problem here is really understanding the Chez arquitecture, rather
> than the RISC-V one with which I have been working for a few years now
> as a consultant.
> 
> The whole point of the work was not to get Chez on RISC-V per se but to
> get Racket on RISC-V. My initial email to the Chez ML was replied to by
> Andy Keep. He replied in great detail on how to create a Chez backend -
> I have cleaned up his reply and have been slowly adding to it [1].
> 
> That was a great start but from there things started to fall apart.
> Further emails to my questions were generally not replied to (of the 4
> messages sent, 1 was replied to) [2].
> 
> Then there is some backend rot... I noticed for example, that there was
> no Makefile for a threaded version of arm32 although the backend file is
> there meaning it should be supported. It seems that it's just that
> nobody every tried to build it. Then software floating point is an
> option in a backend config file but if you enable it, bootstrapping
> doesn't work because the compiler really expects you to have some
> floating point registers.
> 
> Matthew mentions the move to Chez will help maintainability and I am
> sure he's right because he has been working with Racket for a long time
> but my experience comes from looking at backend files. When you look at
> them you end up being forced to look elsewhere, specifically the
> cpnanopass.ss file [3]. Well, this file is the stuff of nightmares...
> It's over 16000 (sixteen thousand!!!) lines of dense scheme code, whose
> comments are not necessarily Chez-Beginner friendly (maybe Alexis wants
> to rewrite it? [4]).
> 
> So I am a bit concerned about this. I somehow get the feeling that
> what's going to happen is that Chez is going to slowly degenerate to a
> Racket sub-project, and nobody is going to really use Chez directly.
> Therefore this means Matthew et al. will end up maintaining it along
> with Racket itself. As far as I understand it, both A. Keep and R.
> Dybvig are both at Cisco and Chez is a side-project from which they are
> slowly distancing themselves. Chez becoming a sub-project of Racket
> might seem far-fetched until you noticed Matthew is already the number 4
> contributor of Chez since it was open-sourced [5].
> 
> The only question I have is really, what do other people feel about
> this. Am I making any sense? Have I missed some hidden Chez community
> that's working day and night into improving it? Or is Chez current sole
> purpose of existence to support Racket?
> 
> [1] https://github.com/LinkiTools/ChezScheme-RISCV/blob/wip-riscv/PORTING.md
> [2]
> https://groups.google.com/forum/#!searchin/chez-scheme/Paulo$20Matos%7Csort:dat
> e
> [3] https://github.com/cisco/ChezScheme/blob/master/s/cpnanopass.ss
> [4] https://twitter.com/lexi_lambda/status/1092539293791330305
> [5] 

[racket-users] Re: Some concern about ChezScheme...

[Greg Trzeciak]

On Tuesday, February 5, 2019 at 2:01:20 PM UTC+1, Paulo Matos wrote:
>
>
> So I am a bit concerned about this. I somehow get the feeling that 
> what's going to happen is that Chez is going to slowly degenerate to a 
> Racket sub-project, and nobody is going to really use Chez directly. 
>
>
... and the two became one...  and from that day they were called RaChez 
[read: ruckus]

I could not resist the pun but hope you will get more meaningful response 
then mine. 
Although if someone is looking for racket2 alternative name, #lang ruckus 
seems fitting (given circumstances).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Tensorflow bindings?

[Stephen De Gabrielle]

Hi all,

Is this relevant?

https://github.com/charlescearl/DeepRacket

Kind regards
Stephen

On Mon, 4 Feb 2019 at 21:39, Neil Van Dyke  wrote:

> I had Racket TensorFlow bindings as a possible-TODO for me, but I don't
> yet know ML tools I'll use, and what APIs/layers I'll prefer atop tools
> that offer more than one.
>
> (One of the reasons to play with ML in Python initially: almost
> everything has Python APIs, at the moment, and some of the most popular
> ML tools are written substantially in Python rather than being a veneer
> of bindings.)
>
> Also, some toolkits might make more sense to call from Racket through
> something other than the current Racket FFI.
>
> (For example, Python ones, or C code you don't want to risk stomping on
> your Racket code, and might not even be open source.  Also, there's also
> a good chance that you're running non-ideal vendor-specific GPU compute
> libraries and drivers right now, though hopefully that will soon improve.)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
-- 


-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Real-world examples of XML usage?

[Konrad Hinsen]

David Storrs  writes:

> to type things.  In addition, most developers that I've worked /
> talked with will typically reach for the JSON API before the XML one
> given the choice.  I think the ground truth suggests that JSON is a

Ah, I see, we are working in very different contexts. In a Web API, I'd
go for JSON as well. After all, you need little more than a
serialization format for the input and output parameters of the API functions.

But my main use case is data storage in self-contained files stored in
databases (including Git repositories), to be read by existing or
yet-to-be-written software. The data can be quite complex, so any
machine-readable description of its structures is a plus. XML on its own
already provides the tag structure, and a schema adds much more
documentation, still in a standard format.

Konrad.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Real-world examples of XML usage?

[David Storrs]

On Tue, Feb 5, 2019 at 2:17 AM Konrad Hinsen  wrote:
>
> David Storrs  writes:
>
>
> > I was specifically thinking of JSON.  It allows for encoding all the
> > essential structure of XML in far fewer characters, meaning there's
> > less data to send over the wire. It's more human-readable. There are
>
> JSON is basically a serialization notation for nested lists and hash
> maps. What it lacks compared to XML is tags and namespaces. You have to
> add a layer of conventions on top of JSON to get some form of data
> identification, and since that layer is not standardized, it's likely to
> get messy once you need to work with data from different sources.
>
> JSON is probably a good choice if all software working with your data is
> under your control. Less so if the client software pool is open-ended or
> if the data may be archived for an indefinite future with different
> software tools.

At the risk of sounding snarky, there are a ton of major services that
provide JSON interfaces to their APIs.  Most provide XML as well, but
people are still going to prefer reading the documentation to poring
over the DTD in order to figure out what's required, optional, and how
to type things.  In addition, most developers that I've worked /
talked with will typically reach for the JSON API before the XML one
given the choice.  I think the ground truth suggests that JSON is a
perfectly fine choice for open-ended services that are being consumed
by software not under your control.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] TFPIE 2019

[Marco Morazan]

Dear All,

It is with great enthusiasm that I write to encourage you to share with the
community of CS educators at large your efforts and experiences. Please
strongly consider submitting a description of your novel work to Trends in
Functional Programming in Education 2019. Below you can find the call for
papers.


TFPIE 2019 Call for papers
https://www.tfp2019.org/index.html
(June 11th, University of British Columbia, Vancouver Canada, co-located
with TFP 2019)

TFPIE 2019 welcomes submissions describing techniques used in the
classroom, tools used
in and/or developed for the classroom and any creative use of functional
programming (FP)
to aid education in or outside Computer Science. Topics of interest
include, but are not
limited to:

  FP and beginning CS students
  FP and Computational Thinking
  FP and Artificial Intelligence
  FP in Robotics
  FP and Music
  Advanced FP for undergraduates
  FP in graduate education
  Engaging students in research using FP
  FP in Programming Languages
  FP in the high school curriculum
  FP as a stepping stone to other CS topics
  FP and Philosophy
  The pedagogy of teaching FP
  FP and e-learning: MOOCs, automated assessment etc.
  Best Lectures Ð more details below

In addition to papers, we are requesting best lecture presentations. WhatÕs
your
best lecture topic in an FP related course? Do you have a fun way to
present FP
concepts to novices or perhaps an especially interesting presentation of a
difficult topic? In either case, please consider sharing it. Best lecture
topics
will be selected for presentation based on a short abstract describing the
lecture and its interest to TFPIE attendees. The length of the presentation
should be comparable to that of a paper. On top of the lecture itself,
the presentation can also provide commentary on the lecture.

Submissions
Potential presenters are invited to submit an extended abstract (4-6 pages)
or a
draft paper (up to 16 pages) in EPTCS style. The authors of accepted
presentations will have their preprints and their slides made available on
the
workshop's website. Papers and abstracts can be submitted via easychair at
the
following link: https://easychair.org/conferences/?conf=tfpie2019 After the
workshop, presenters will be invited to submit (a revised version of) their
article for review. The PC will select the best articles that will be
published in the Electronic Proceedings in Theoretical Computer Science
(EPTCS).
Articles rejected for presentation and extended abstracts will not be
formally
reviewed by the PC.

Dates
Submission deadline: May 14th 2019, Anywhere on Earth.
Notification: May 20th
Workshop: June 11th
Submission for formal review: August 18th 2019, Anywhere on Earth
Notification of full article: October 6th
Camera ready: November 1st


Program Committee

Alex Gerdes - University of Gothenburg / Chalmers
Jurriaan Hage (Chair) - Utrecht University
Pieter Koopman - Radboud University, the Netherlands
Elena Machkasova - University of Minnesota, Morris, USA
Heather Miller - Carnegie Mellon University and EPFL Lausanne
Prabhakar Ragde - University of Waterloo, Waterloo, Ontario, Canada
Simon Thompson - University of Kent, UK
Sharon Tuttle - Humboldt State University, Arcata, USA

Note: information on TFP is available at https://www.tfp2019.org/index.html


-- 

Cheers,

Marco

Have a´¨)
¸.·´¸.·*´¨) ¸.·*¨)
(¸.·´ (¸.·´ * wonderful day! :)

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Re: updated Racket-on-Chez status

[Gustavo Massaccesi]

 I have been trying a few variations of the code. It would be nice to have
a test branch that use only the data in the repository. I used some fake
data instead.

For the tests, I used the function *get-mean-max-bounds*
https://github.com/alex-hhh/ActivityLog2/blob/master/rkt/data-frame/meanmax.rkt#L409
with this data

  (define fake-data2
(for/list ([_ (in-range 1000)])
  (if (< (random) .01)
 (vector #f #f)
 (vector (- (random) .5) (- (random) .5)

so, I tested with

  (time (get-mean-max-bounds fake-data2))



*** The main time improvement was changing
  (for ([b bavg] #:when (vector-ref b 1))
...)
to
  (for ([b (*in-list* bavg)] #:when (vector-ref b 1))
...)

This increase the speed to the double or more. In the microbenchmark, the
new duration is the 40%-50% of the original duration.

IIUC, in all functions you know the type of sequence of the arguments, so
my advice is to add in-list or in-vector to each and every for in the whole
file (or project).

This is a good general recommendation. With in-list or in-vector or
in-range, the generated code is very efficient. Without them, the code has
to create a generic object to track the iteration, and the code is much
slower.


*** I tried eliminating the set! and using for/fold instead. The problem is
that the code is slower :(. In general it's better to avoid mutable
variables, but in this case removing them makes the program slower. We
should take a look at the internal code of Racket and try to fix it,
because in a perfect world the version without set! should be faster.
Meanwhile, keep the current version...


*** I tried replacing the for and set! with an explicit loop. Something like
 (let loop ([bavg bavg] [min-x #f] [max-x #f]  [min-y #f] [max-y #f])
   ...)

With this change, there is an additional 5% improvement in the speed, but
the legibility is reduced too much. So this is better than the version with
for and in-list, but I recommend to keep the legible version.


*** I tried replacing the initial value of min-x and friends with +inf.0,
and removing the if in the updates. I'm convinced this is a good idea, but
the change in speed is negligible.




In conclusion, try adding as much in-list, in-vector and in-range as you
can.

Gustavo




On Thu, Jan 31, 2019 at 9:58 AM Alex Harsanyi 
wrote:

>
> On Thursday, January 31, 2019 at 9:23:39 AM UTC+8, Matthew Flatt wrote:
>>
>> > I would be happy to help you identify where the performance degradation
>> > between Racket 7.1 and CS is when running these tests.
>>
>> Small examples that illustrate slowness in a specific subsystem are
>> always helpful. I can't always make the subsystem go faster right away,
>> but sometimes.
>>
>>
> I timed some key functions in my application to understand which parts of
> Racket CS are slow.  I did a write-up in the Gist listed below, but the
> result seems to be that even functions that run Racket only code with no IO
> or calls into C libraries run slower in Racket CS.  Code that calls into
> the database library to run SQL insert queries runs significantly slower.
> The only things which were faster in Racket CS were one "Racket only"
> function, `df-histogram` and a function which retrieved data from an SQL
> query, `df-read/sql`
>
> https://gist.github.com/alex-hhh/1ebc1c83b68ee4620a70fc30d2caa6a3
>
> Alex.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Some concern about ChezScheme...

['Paulo Matos' via Racket Users]

Hi all,

Now that I got your attention... :)
Although the title is not purely click-bait, it is motivated by personal
requirements.

Most of us are happy with the move to Chez (actually haven't heard
anyone opposing it), but I would like to point to something I have felt
over the past year and to understand if this is just my feeling or not
from others (maybe Matthew) who have had any experience working with Chez.

I have been, for over a year on and off, trying to port Chez to RISC-V.
My problem here is really understanding the Chez arquitecture, rather
than the RISC-V one with which I have been working for a few years now
as a consultant.

The whole point of the work was not to get Chez on RISC-V per se but to
get Racket on RISC-V. My initial email to the Chez ML was replied to by
Andy Keep. He replied in great detail on how to create a Chez backend -
I have cleaned up his reply and have been slowly adding to it [1].

That was a great start but from there things started to fall apart.
Further emails to my questions were generally not replied to (of the 4
messages sent, 1 was replied to) [2].

Then there is some backend rot... I noticed for example, that there was
no Makefile for a threaded version of arm32 although the backend file is
there meaning it should be supported. It seems that it's just that
nobody every tried to build it. Then software floating point is an
option in a backend config file but if you enable it, bootstrapping
doesn't work because the compiler really expects you to have some
floating point registers.

Matthew mentions the move to Chez will help maintainability and I am
sure he's right because he has been working with Racket for a long time
but my experience comes from looking at backend files. When you look at
them you end up being forced to look elsewhere, specifically the
cpnanopass.ss file [3]. Well, this file is the stuff of nightmares...
It's over 16000 (sixteen thousand!!!) lines of dense scheme code, whose
comments are not necessarily Chez-Beginner friendly (maybe Alexis wants
to rewrite it? [4]).

So I am a bit concerned about this. I somehow get the feeling that
what's going to happen is that Chez is going to slowly degenerate to a
Racket sub-project, and nobody is going to really use Chez directly.
Therefore this means Matthew et al. will end up maintaining it along
with Racket itself. As far as I understand it, both A. Keep and R.
Dybvig are both at Cisco and Chez is a side-project from which they are
slowly distancing themselves. Chez becoming a sub-project of Racket
might seem far-fetched until you noticed Matthew is already the number 4
contributor of Chez since it was open-sourced [5].

The only question I have is really, what do other people feel about
this. Am I making any sense? Have I missed some hidden Chez community
that's working day and night into improving it? Or is Chez current sole
purpose of existence to support Racket?

[1] https://github.com/LinkiTools/ChezScheme-RISCV/blob/wip-riscv/PORTING.md
[2]
https://groups.google.com/forum/#!searchin/chez-scheme/Paulo$20Matos%7Csort:date
[3] https://github.com/cisco/ChezScheme/blob/master/s/cpnanopass.ss
[4] https://twitter.com/lexi_lambda/status/1092539293791330305
[5] https://github.com/cisco/ChezScheme/graphs/contributors

-- 
Paulo Matos

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Quickscript error on first startup of Racket 7.2

[Robby Findler]

Sounds like you are already in good shape!

Robby

On Tue, Feb 5, 2019 at 12:57 AM Laurent  wrote:

> On Mon, Feb 4, 2019 at 9:55 PM Robby Findler 
> wrote:
>
>> > Possibly. I'm not sure what would be the best option yet, I'll think
>> about it but I welcome suggestions.
>>
>> I think the main thing should be to avoid any uncaught exceptions (if
>> they happen while quickscript starts up, it will get disabled; if they
>> happen later, they'll be labelled "internal error"s). How exactly to
>> save the information and show it to the user in a nice way seems like
>> there are lots of things to try out!
>>
>
> Currently, exceptions raised from building the menu or running the scripts
> are caught by quickscript and displayed in a message box. I've just changed
> the title of the box to "Quickscript caught an exception" to be clearer.
> Another option would be to have a quickscript log accessible from the menu
> (and a quickscript console?).
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] [TFP'19] first call for papers: Trends in Functional Programming 2019, 12-14 June 2019, Vancouver, BC, CA

[p.achten]

 ---
   C A L L  F O R  P A P E R S
 ---

  == TFP 2019 ==

  20th Symposium on Trends in Functional Programming
   12-14 June, 2019
  Vancouver, BC, CA
  https://www.tfp2019.org/index.html


== Important Dates ==

Submission DeadlineThursday, March 28, 2019
Paper Notification Thursday, May 2, 2019
TFPIE  Tuesday, June 11, 2019
Symposium  Wednesday, June 12, 2019 – Friday, June 14, 
2019
Student Paper Feedback Friday June 21, 2019
Submission for Formal Review   Thursday, August 1, 2019
Notification of Acceptance Thursday, October 24, 2019
Camera Ready   Friday, November 29, 2019


The symposium on Trends in Functional Programming (TFP) is an
international forum for researchers with interests in all aspects of
functional programming, taking a broad view of current and future
trends in the area. It aspires to be a lively environment for
presenting the latest research results, and other contributions (see
below at scope). 

Please be aware that TFP uses two distinct rounds of submissions (see
below at submission details).

TFP 2019 will be the main event of a pair of functional programming
events. TFP 2019 will be accompanied by the International Workshop on
Trends in Functional Programming in Education (TFPIE), which will take
place on June 11.


== Scope ==

The symposium recognizes that new trends may arise through various routes. 
As part of the Symposium's focus on trends we therefore identify the 
following five article categories. High-quality articles are solicited in 
any of these categories:

Research Articles:
Leading-edge, previously unpublished research work
Position Articles:
On what new trends should or should not be
Project Articles:
Descriptions of recently started new projects
Evaluation Articles:
What lessons can be drawn from a finished project
Overview Articles:
Summarizing work with respect to a trendy subject

Articles must be original and not simultaneously submitted for publication 
to any other forum. They may consider any aspect of functional programming: 
theoretical, implementation-oriented, or experience-oriented. Applications 
of functional programming techniques to other languages are also within the 
scope of the symposium.

Topics suitable for the symposium include, but are not limited to:

Functional programming and multicore/manycore computing
Functional programming in the cloud
High performance functional computing
Extra-functional (behavioural) properties of functional programs
Dependently typed functional programming
Validation and verification of functional programs
Debugging and profiling for functional languages
Functional programming in different application areas:
security, mobility, telecommunications applications, embedded
systems, global computing, grids, etc.
Interoperability with imperative programming languages
Novel memory management techniques
Program analysis and transformation techniques
Empirical performance studies
Abstract/virtual machines and compilers for functional languages
(Embedded) domain specific languages
New implementation strategies
Any new emerging trend in the functional programming area

If you are in doubt on whether your article is within the scope of TFP, 
please contact the TFP 2019 program chairs, William J. Bowman and Ron 
Garcia.


== Best Paper Awards ==

To reward excellent contributions, TFP awards a prize for the best paper 
accepted for the formal proceedings.

TFP traditionally pays special attention to research students, 
acknowledging 
that students are almost by definition part of new subject trends. A 
student 
paper is one for which the authors state that the paper is mainly the work 
of 
students, the students are listed as first authors, and a student would 
present 
the paper. A prize for the best student paper is awarded each year.

In both cases, it is the PC of TFP that awards the prize. In case the best 
paper happens to be a student paper, that paper will then receive both 
prizes.


== Instructions to Author ==

Papers must be submitted at:

https://easychair.org/conferences/?conf=tfp2019

Authors of papers have the choice of having their contributions formally 
reviewed either before or after the Symposium.


== Pre-symposium formal review ==

Papers to be formally reviewed before the symposium should be submitted 
before an early deadline and receive their reviews and notification of 
acceptance for both presentation and publication before the symposium. A 
paper that has been rejected in this process may still be accepted for 
presentation at the symposium, but will not be