Re: [racket-users] reducing pauses with incremental GC
Wow, this is great. I just tried it with one of my games that I made with big-bang, and the pauses I was having are completely gone, as far as I can tell. (I was doing quite a bit of copying, so I think the GC was triggering fairly often.) Thanks a lot for this; it’s definitely a big improvement. > On Dec 2, 2015, at 7:44 AM, Matthew Flatt wrote: > > The development version of Racket now supports an incremental > garbage-collection mode that can eliminate long pauses in a program. > For example, incremental mode is useful for avoiding pauses in games > and animations. > > For more information, see > > http://goo.gl/rYjTIi > > You can try out incremental mode in a snapshot build: > > http://pre.racket-lang.org/ > > > The `2htdp/universe` library has been updated to request incremental > mode, so a `big-bang` program will automatically use it. The programs > that I've tried experience GC pauses that are bounded by 10ms or so > (when run outside of DrRacket). In v6.3, the same programs see a pair > of 130ms pauses every 2-3 minutes. > > As another example, my son Oliver has a Pict3D-based game that now runs > without big pauses: > > https://spaceorbs.weebly.com/ > https://www.youtube.com/watch?v=mP8ud9Yztz8 > > The game uses a fork of Pict3D (github.com/mflatt/pict3d) where I've > added a `(collect-garbage 'incremental)` call. That changes also makes > animations in my Pict3D-based talk run without big pauses. > > For Oliver's game, each GC pause is usually below 16ms on his machine, > but sometimes between 16ms and 32ms. As a result, the game is still > dropping frames, so it's not yet as good as we want --- but that's much > better than before, and the game is comfortably playable. Further > improvement might be just a matter of tuning the implementation of > incremental mode. > > > There's still room for improvement overall, and your mileage may vary. > If you have an application that needs shorter GC pauses and the current > implementation of incremental mode doesn't work well enough, I'm > interested to hear about it. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] Windows show-scrollbars forcing call to on-paint
Short version: show-scrollbars seems to be forcing my canvas object to call on-paint. The problem is that on-paint contains calls to show-scrollbars and thus creates an infinite loop. Long version: I have a canvas that displays an image and will enable or disable the scrollbars based on how large the image is in relation to the current dimensions of the canvas. On Linux and OSX the code operates as intended and the user may resize the frame as much as desired with the scrollbars appearing or disappearing when appropriate. On Windows, however, simply getting the canvas to display the image will trap itself inside an infinite loop because the calls to show-scrollbars is making the canvas call on-paint. I have narrowed it down to specifically that procedure because removing those calls from on-paint allows the canvas to display the image properly. I have tried all kinds of combinations of suspend- and resume-flush but nothing seems to change this behavior. For reference, the relevant lines may be found here: https://github.com/lehitoskin/ivy/blob/master/base.rkt#L264-L310 -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Re: [racket] How to extend image-snip% with a new field
Ah, right: that code didn't actually save the bitmap itself. How do you like this code? Robby #lang racket/gui (require (prefix-in - racket/base)) (provide (rename-out [image-data-snip-class snip-class]) image-data-snip%) (define image-data-snip% (class image-snip% (inherit set-snipclass get-flags set-flags get-admin get-extent draw) (define metadata (make-hash)) (define/public (set-metadata md) (set! metadata md)) (super-new) (set-snipclass image-data-snip-class) (set-flags (cons 'handles-events (get-flags))) (inherit get-bitmap) (define/override (copy) (define ids (new image-data-snip%)) (send ids set-bitmap (get-bitmap)) (send ids set-metadata metadata) ids) (define/override (write f) (define bmp (get-bitmap)) (define w (send bmp get-width)) (define h (send bmp get-height) ) (define p (open-output-bytes)) (-write (list metadata w h) p) (define b (get-output-bytes p)) (define ib (make-bytes (* 4 w h))) (send bmp get-argb-pixels 0 0 w h ib) (send f put (bytes-length b) b) (send f put (bytes-length ib) ib)) ;;Operations on the metadata ;;; (define/public (get-arg-pos arg) (hash-ref metadata arg '(0.5 0.5))) (define/public (set-arg-pos arg pos) (hash-set! metadata arg pos)) (define/public (get-args-pos) (hash->list metadata (define image-data-snip-class% (class snip-class% (inherit set-classname) (super-new) (set-classname (~s '(lib "main.rkt" "image-data-snip"))) (define/override (read f) (match-define (list md w h) (-read (open-input-bytes (send f get-unterminated-bytes (define img-bytes (send f get-unterminated-bytes)) (define bmp (make-bitmap w h)) (send bmp set-argb-pixels 0 0 w h img-bytes) (define ids (new image-data-snip%)) (send ids set-metadata md) (send ids set-bitmap bmp) ids))) (define image-data-snip-class (new image-data-snip-class%)) (send (get-the-snip-class-list) add image-data-snip-class) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] web-server dispatch and places
On 12/2/2015 7:47 AM, Jay McCarthy wrote: Hi George, The output port (and input port) that a response gets is the real port, meaning that it has a file-descriptor and can be shared across places. I suggest having one serve/servlet place that does the dispatching, forwards the port (perhaps using response/output if you know the headers and if not, using a new function that I can help you write) over to another place depending on what the API call is, that other place does the actual writing, the original place waits for the completion and then returns (thus closing/recycling the connection). I'd be happy to look at your code and point you to the internal functions you might need to write out the response properly. Jay Hi Jay, Thanks for confirming about the port, and also for the offer of help. In almost all cases the return headers are static: almost all server->client communication is JSON formatted - regardless of the disposition of the function call , its wrapping HTTP request succeeds. Errors (if any) are returned in the JSON data. But there is one function which returns binary image files. It's designed to mimic file download so it can be used directly in an HTML image tag. Might need a spot of help with that one. It currently looks like: : ; DBMS fetch image file into 'result' : (send/back (if success (response/full 200 #"OK" (current-seconds) #"application/octet-stream" '() `(,result)) (response/full 404 #"REQUESTED OBJECT NOT FOUND" (current-seconds) #"application/text" '() '()) )) The only other potential difficulty I see would be if a place(d) function needed to set a cookie on the client. At least for now, none of the functions I would want to separate need to do that. I'm going to have to investigate places more and develop a framework on which to build my (core) distributed version. I have to address some things like sharing log files [relatively easy I think] and maybe sharing of a DBMS connection pool. It's an open question whether it's better to share a connection pool (if possible) among a small number of multi-threaded places, or to create a large (pool equivalent) number of single-threaded places each keeping one connection open. George -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Re: reducing pauses with incremental GC
I think, always. Because they like games too. On Thu, Dec 3, 2015 at 2:35 AM, Jack Firth wrote: > Absolutely fantastic! I wonder how often language designers implement > features useful to game developers because of their children. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] Re: reducing pauses with incremental GC
On Wednesday, December 2, 2015 at 10:44:47 AM UTC-5, Matthew Flatt wrote: > The development version of Racket now supports an incremental > garbage-collection mode that can eliminate long pauses in a program. > For example, incremental mode is useful for avoiding pauses in games > and animations. By chance then will the Dr. Racket environment itself honor the PLT_INCREMENTAL_GC variable and perhaps benefit from the less noticeable GC? Thanks for a good addition. JG -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Re: [racket] How to extend image-snip% with a new field
quarta-feira, 2 de Dezembro de 2015 às 13:34:21 UTC, Robby Findler escreveu: > I think this code is doing what you want. > > Robby > > #lang racket/gui > > (require (prefix-in - racket/base)) > (provide (rename-out [image-data-snip-class snip-class]) > image-data-snip%) > > (define image-data-snip% > (class image-snip% > (inherit set-snipclass > get-flags set-flags > get-admin > get-extent > draw) > > (define metadata (make-hash)) > (define/public (set-metadata md) (set! metadata md)) > (super-new) > > (set-snipclass image-data-snip-class) > (set-flags (cons 'handles-events (get-flags))) > > (define/override (copy) > (define ids (new image-data-snip%)) > (send ids set-metadata metadata) > ids) > > (define/override (write f) > (define p (open-output-bytes)) > (-write metadata p) > (define b (get-output-bytes p)) > (send f put (bytes-length b) b)) > > > ;;Operations on the metadata ;;; > > > (define/public (get-arg-pos arg) > (hash-ref metadata arg '(0.5 0.5))) > > (define/public (set-arg-pos arg pos) > (hash-set! metadata arg pos)) > > (define/public (get-args-pos) > (hash->list metadata > > (define image-data-snip-class% > (class snip-class% > (inherit set-classname) > > (super-new) > (set-classname (~s '(lib "main.rkt" "image-data-snip"))) > > (define/override (read f) > (define byts (send f get-unterminated-bytes)) > (define ids (new image-data-snip%)) > (send ids set-metadata (-read (open-input-bytes byts))) > ids))) > > (define image-data-snip-class (new image-data-snip-class%)) > (send (get-the-snip-class-list) add image-data-snip-class) Thanks Robby, for the reply, this code do almost everything that I want. The missing part is to display the bitmap when the object is created. For example, the following evocation returns an empty image >> (define ids (make-object image-data-snip% "path/to/img.png")) >> ids |x| But if I try.. >> (send ids get-bitmap) It returns the filled bitmap. I tried to rewrite the draw method, but in the subclass I don't have the bitmap yet unless I store it there. However, it seems awkward to store the bitmap in this class again. Can you tell me what is the right way to solve this problem. Thank you very much, and sorry for my inconvenience. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Help with a bewildering bug.
As a follow-up for the list, it looks like the issue was resolved following a discussion on the github issue. Vincent On Wed, 02 Dec 2015 03:25:55 -0600, John Berry wrote: > > I've been working on a new feature for Heresy and I've run into a bug I > cannot even begin to track down the actual cause of. > > There's a Github issue here: https://github.com/jarcane/heresy/issues/37 > > I've tried a few naive solutions, but to be honest I really don't even > know where to start because even the errors I get aren't even very > informative. Run locally in the monadish module the code returns an > error with line number and highlights the for loop in :>, run in another > file I've had errors flag on both #%app and make-thing in the mentioned > modules in the Github issue. > > Any pointers would be rather appreciated, I suspect there's some simple > abuse of scope or hygiene I'm missing but I'm having trouble following > the layers and layers of abstraction involved here. > > -- > You received this message because you are subscribed to the Google > Groups "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] Re: reducing pauses with incremental GC
Absolutely fantastic! I wonder how often language designers implement features useful to game developers because of their children. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] reducing pauses with incremental GC
This is awesome, Matthew. Best Christmas ever! Neil V. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] reducing pauses with incremental GC
At Wed, 2 Dec 2015 11:26:05 -0500, Matt Jadud wrote: > "Dad, when are you going to push that incremental GC? I want to move > Space Orbs out of alpha..." Exactly. And there was a game before Space Orbs. He's been waiting for a couple of years. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] reducing pauses with incremental GC
I took a look at the server code, and it made my heart warm to see HtDP patterns so clearly/cleanly expressed. Very cool. I've also been having fun imagining the dinner table conversations... "Dad, when are you going to push that incremental GC? I want to move Space Orbs out of alpha..." Nice work all around. Cheers, M On Wed, Dec 2, 2015 at 10:44 AM, Matthew Flatt wrote: > The development version of Racket now supports an incremental > garbage-collection mode that can eliminate long pauses in a program. > For example, incremental mode is useful for avoiding pauses in games > and animations. > > For more information, see > > http://goo.gl/rYjTIi > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] reducing pauses with incremental GC
Awesome Matthew! Jay On Wed, Dec 2, 2015 at 10:44 AM, Matthew Flatt wrote: > The development version of Racket now supports an incremental > garbage-collection mode that can eliminate long pauses in a program. > For example, incremental mode is useful for avoiding pauses in games > and animations. > > For more information, see > > http://goo.gl/rYjTIi > > You can try out incremental mode in a snapshot build: > > http://pre.racket-lang.org/ > > > The `2htdp/universe` library has been updated to request incremental > mode, so a `big-bang` program will automatically use it. The programs > that I've tried experience GC pauses that are bounded by 10ms or so > (when run outside of DrRacket). In v6.3, the same programs see a pair > of 130ms pauses every 2-3 minutes. > > As another example, my son Oliver has a Pict3D-based game that now runs > without big pauses: > > https://spaceorbs.weebly.com/ > https://www.youtube.com/watch?v=mP8ud9Yztz8 > > The game uses a fork of Pict3D (github.com/mflatt/pict3d) where I've > added a `(collect-garbage 'incremental)` call. That changes also makes > animations in my Pict3D-based talk run without big pauses. > > For Oliver's game, each GC pause is usually below 16ms on his machine, > but sometimes between 16ms and 32ms. As a result, the game is still > dropping frames, so it's not yet as good as we want --- but that's much > better than before, and the game is comfortably playable. Further > improvement might be just a matter of tuning the implementation of > incremental mode. > > > There's still room for improvement overall, and your mileage may vary. > If you have an application that needs shorter GC pauses and the current > implementation of incremental mode doesn't work well enough, I'm > interested to hear about it. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Jay McCarthy Associate Professor PLT @ CS @ UMass Lowell http://jeapostrophe.github.io "Wherefore, be not weary in well-doing, for ye are laying the foundation of a great work. And out of small things proceedeth that which is great." - D&C 64:33 -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] reducing pauses with incremental GC
The development version of Racket now supports an incremental garbage-collection mode that can eliminate long pauses in a program. For example, incremental mode is useful for avoiding pauses in games and animations. For more information, see http://goo.gl/rYjTIi You can try out incremental mode in a snapshot build: http://pre.racket-lang.org/ The `2htdp/universe` library has been updated to request incremental mode, so a `big-bang` program will automatically use it. The programs that I've tried experience GC pauses that are bounded by 10ms or so (when run outside of DrRacket). In v6.3, the same programs see a pair of 130ms pauses every 2-3 minutes. As another example, my son Oliver has a Pict3D-based game that now runs without big pauses: https://spaceorbs.weebly.com/ https://www.youtube.com/watch?v=mP8ud9Yztz8 The game uses a fork of Pict3D (github.com/mflatt/pict3d) where I've added a `(collect-garbage 'incremental)` call. That changes also makes animations in my Pict3D-based talk run without big pauses. For Oliver's game, each GC pause is usually below 16ms on his machine, but sometimes between 16ms and 32ms. As a result, the game is still dropping frames, so it's not yet as good as we want --- but that's much better than before, and the game is comfortably playable. Further improvement might be just a matter of tuning the implementation of incremental mode. There's still room for improvement overall, and your mileage may vary. If you have an application that needs shorter GC pauses and the current implementation of incremental mode doesn't work well enough, I'm interested to hear about it. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] [racket] Racket 6.3(.0.7) breaks old code
Thanks for the example! The problem is a bug in `namespace-mapped-symbols` (due to changes related to the new macro expander). I will push a repair later today. At Mon, 30 Nov 2015 20:16:44 +0800, WarGrey Gyoudmon Ju wrote: > Thank you, Matthew. > > The project itself is organized as a multi package, however there is no > need to install it. > > Debug/DigiGnome ; this subpackage is the base one of the other two. > Debug/Kuzuhamon/; this one meets the second problem > Debug/sakuyamon/ ; this one meets the first problem > Debug/sakuyamon/digitama/dispatch.rkt ; it is the buggy source > Debug/Kuzuhamon/digivice/land-bang.rkt ; it is the buggy source, the bug > only occurs after building > Debug/DigiGnome/makefile.rkt; it builds the entire > project with (compile-directory-zos) and info.rkt > Debug/DigiGnome/digitama/digicore.rkt ; it is the entry module (but > not the application launcher) of all > Debug/DigiGnome/digitama/posix.rkt ; > (module-prefab:cstruct/no-auto-update) is here. > > digitama means 'private', while digivice means 'bin'. > to build the project, just `cd` into the target directory DigiGnome, > sakuyamon, or Kuzuhamon, then run `../DigiGnome/makefile.rkt`. > > (module-prefab:cstruct/no-auto-update) defines a submodule which contains > another submodule in it. The purpose is defining prefab structs based on c > structs for both untyped and typed racket. Currently it is used in > sakuyamon/digitama/posix.rkt to get the system stats. > > dispatch.rkt always stops the building process (and `racket`), makefile.rkt > itself is all right with this minimal code base, actually it just fails > following the other failures. weird. > > So... if there is something wrong with the compiler (makefile.rkt itself is > also compiled)? > without compiling, digivice/land-bang.rkt can load the games (with > in Kuzuhamon/village/land-of-lisp) correctly. > > > On Mon, Nov 30, 2015 at 8:54 AM, Matthew Flatt wrote: > > > My guess is that you're running into an incompatibility created by the > > new macro system. Something like lifting code out of an expanded > > `module` form and dropping it into a different one? A change related to > > submodule expansion? Or something related to the top-level namespace? > > It might be an unavoidable incompatibility, or it might be a bug. > > > > I'm unclear on what I'd need to do to replicate the problem, though. Is > > there some code of yours that's available and that I could try to run? > > > > At Mon, 30 Nov 2015 06:53:28 +0800, WarGrey Gyoudmon Ju wrote: > > > It is compiled by myself with no strange options. > > > Rebuilding the project (without DrRacket) fails due to two strange > > > behaviors. > > > > > > The first one is module-path-index-resolve: "self" index has no > > resolution. > > > It breaks lots of scripts, I cannot locate where it is exactly occurs. > > > 1. errortrace says makefile.rkt:364:35: (namespace-mapped-symbols) > > > this is my building facility written in untyped racket, the > > stack > > > point is parameterized with (variable-reference->namespace > > > (#%variable-reference)). > > > > > > 2. errortrace says nothing, while `racket` puts > > > module-path-index-resolve: "self" index has no resolution > > > module path index: # > > > context...: > > >/opt/PLTracket/collects/syntax/private/id-table.rkt:77:2: do-ref > > > > > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/rep/interning.rkt:22:13 > > > : > > > *Opaque > > >(submod .../digitama/posix.rkt typed/ffi #%type-decl): [running body] > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/env/env-req.rkt:8:4: > > > for-loop > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/tc-setup.rkt:82:0: > > > tc-module/full > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/typed-racket.rkt:24:4 > > >standard-module-name-resolver > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/tc-setup.rkt:82:0: > > > tc-module/full > > > > > > > > /opt/PLTracket/share/pkgs/typed-racket-lib/typed-racket/typed-racket.rkt:24:4 > > >standard-module-name-resolver > > > > > > this is a normal module, only posix.rkt is written by me, > > > but... `racket posix.rkt` is okay, in which case, ffi bindings are > > written > > > in the top level module, typed/ffi is its submodule, `racket` run the > > > tests based on the typed one. DrRacket complains > > > (current-load-relative-directory) > > > is false. > > > > > > > > > > > > The second one is compiled/expanded code out of context; cannot find > > > exports to restore imported renamings for module: ... > > > It seems that (dynamic-require) does not work with the correct > > > (current-directory) or (current-load-relative-directory). > > > > > > Thanks. > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Racket Users" group. > > > To u
Re: [racket-users] Re: [racket] How to extend image-snip% with a new field
I think this code is doing what you want. Robby #lang racket/gui (require (prefix-in - racket/base)) (provide (rename-out [image-data-snip-class snip-class]) image-data-snip%) (define image-data-snip% (class image-snip% (inherit set-snipclass get-flags set-flags get-admin get-extent draw) (define metadata (make-hash)) (define/public (set-metadata md) (set! metadata md)) (super-new) (set-snipclass image-data-snip-class) (set-flags (cons 'handles-events (get-flags))) (define/override (copy) (define ids (new image-data-snip%)) (send ids set-metadata metadata) ids) (define/override (write f) (define p (open-output-bytes)) (-write metadata p) (define b (get-output-bytes p)) (send f put (bytes-length b) b)) ;;Operations on the metadata ;;; (define/public (get-arg-pos arg) (hash-ref metadata arg '(0.5 0.5))) (define/public (set-arg-pos arg pos) (hash-set! metadata arg pos)) (define/public (get-args-pos) (hash->list metadata (define image-data-snip-class% (class snip-class% (inherit set-classname) (super-new) (set-classname (~s '(lib "main.rkt" "image-data-snip"))) (define/override (read f) (define byts (send f get-unterminated-bytes)) (define ids (new image-data-snip%)) (send ids set-metadata (-read (open-input-bytes byts))) ids))) (define image-data-snip-class (new image-data-snip-class%)) (send (get-the-snip-class-list) add image-data-snip-class) -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] "bad variable linkage" after restarting handin server under load
Thanks: I've pushed the syncronization code to the handin repo.
And yes, you're right that `thread` (which is what I think the handin
server uses) doesn't make use of multiple cores. Places sound like the
right construct to use here, but I can already predict one problem:
2htdp/universe depends on racket/gui so it can't be run except in the
main place.
My best guess at a solution would be to provide an alternative version
of 2htdp/universe that doesn't depend on racket/gui (and so doesn't
actually open any windows or do any GUI stuff but instead either
raises errors or simply simulates the tick handler only) and then set
up a namespace for evaluating student programs in another place that
has that dummy version of 2htdp/universe. (I think that the dummy
version of 2htdp/universe would be useful even without different
places for running tests on student code so the server doesn't have
windows popping up anyway!).
That change is a more significant change, however, and would probably
take the form of a change to the handin server that set up the
separate places. I'd be happy to provide advice if you want to look
into this change. I don't think it will be too hard, but it will
require actual work. Or I could put it on a list of things I want to
work on (that sadly doesn't seem to ever shrink...)
I imagine it would be possible to do this in the checker itself if you
wanted to experiment there. (That wouldn't help with the
2htdp/universe dependency problem, however.)
Providing an alternate version of 2htdp/universe is also in the
category of not-too-hard, but requiring-work. Mostly, I guess,
refactoring the library to move dependencies around and then building
a simple layer on top of the refactored code that avoids the
racket/gui dependency. (Or maybe it's already factored well!)
Robby
On Tue, Dec 1, 2015 at 6:55 AM, Paolo Giarrusso wrote:
> Hi!
> After a new deadline, I got good news and bad news.
>
> # Good news
>
> I think *this* bug is fixed. Evidence: instead of crashing at the
> first reboot under load, the server survived to 10-20 automated
> reboots with the students submitting en masse without never showing
> the bug. So not only the patch makes sense, but it seems to be for the
> same bug.
>
> # Bad news
>
> The setup still didn't scale, though this wasn't as bad, and part of
> it was due to my setup. One student compared it to new releases from
> Blizzard. While our beefy server isn't even remotely sweating O_O.
>
> So I'd like to understand Racket threads and the handin server, to
> plan accordingly:
>
> - Does the whole handin server actually run on *one* processor,
> because of Racket multithreading?
> - What's your largest deployment with active checkers?
> - Do you actually use this for HtDP courses, or only for advanced
> classes (as a number of signs suggest)?
>
> 1. Under load, requesting the home page takes more than 20 seconds, so
> my watchdog scripts restarts the server. We have a watchdog script
> because when we didn't, the server just hung sometimes, so for our
> previous lecture (smaller, only ~100 students instead of 500, and no
> checkers) this watchdog script did wonders.
> 2. Here's the watchdog:
> curl --max-time 20 -s
> https://handin-ps.informatik.uni-tuebingen.de:7979/ > /dev/null || {
> docker restart handin-server-production; }
> That's even running every minute :-(
>
> Usually that request takes 20 ms, so (naive me thought) how on Earth
> could this balloon to 20 seconds?
> Now that I know of Racket threads, I understand: that includes both
> the web server and the checkers, together with an unspecified number
> of big-bang instances from students. For extra fun, one students
> called animate with big-bang as step function — essentially, a sweet
> HtDP fork bomb.
>
> I don't expect a patch for this, I'm just trying to understand things
> and contemplating workarounds, beyond a more lenient watchdog (or
> disabling it altogether and acting by hand), which I guess won't be
> enough.
>
> Cheers,
> Paolo
>
> On 29 November 2015 at 16:12, Robby Findler
> wrote:
>>
>>
>> On Sunday, November 29, 2015, Paolo Giarrusso wrote:
>>>
>>> On Friday, November 27, 2015 at 3:44:20 AM UTC+1, Robby Findler wrote:
>>> > Yes, I think you're right. I originally wrote that because I was
>>> > thinking that this code might be involved in evaluating the user's
>>> > submission, but I am not pretty sure I was wrong about that.
>>>
>>> "not pretty sure"?
>>
>>
>> Sorry. No "not".
>>
>>
>>>
>>>
>>> AFAICS, `auto-reload-value` is used to extract the `checker` binding from
>>> the various checker.rkt. but the lock will not be held while running
>>> `checker`. (Luckily we're not using hooks, I haven't studied that code).
>>
>>
>> Yes that's also what I noticed and why I sent a second diff. Or did I miss
>> another place?
>
> Was just rechecking because of the above confusion. We agree.
>
> --
> Paolo G. Giarrusso - Ph.D. Student, Tübingen University
> http://ps.informatik.uni-tuebingen.de/te
Re: [racket-users] web-server dispatch and places
Hi George, The output port (and input port) that a response gets is the real port, meaning that it has a file-descriptor and can be shared across places. I suggest having one serve/servlet place that does the dispatching, forwards the port (perhaps using response/output if you know the headers and if not, using a new function that I can help you write) over to another place depending on what the API call is, that other place does the actual writing, the original place waits for the completion and then returns (thus closing/recycling the connection). I'd be happy to look at your code and point you to the internal functions you might need to write out the response properly. Jay On Wed, Dec 2, 2015 at 1:08 AM, George Neuner wrote: > On 12/1/2015 9:51 PM, Robby Findler wrote: > >> You probably thought of this, but could you hand the ports themselves >> over to the worker places after the dispatch but before any other >> processing happened? Or maybe just make each place run the entire >> server and then hand the ports over before dispatching and dispatch >> inside each worker place? >> > > Hi Robby, > > I didn't think of that exactly. Using serve/servlet, I don't know how to > get the port before I wind up in a handler function. Once inside a > handler, AFAIK, the port is only accessible from response/output (or a > custom response function which I haven't attempted). > > My initial idea was to have each place run serve/servlet using its own set > of dispatch rules. I was thinking about having one place handle all the > low volume requests plus some background tasks, and having a number of > identical places handle the high volume requests. > > I did think of having just one place perform dispatch, and using handler > stubs such as: > > (define (high-volume-function request) > (response/output > (lambda (port) > (let [(ch (get-a-worker)) > (msg (hash 'req request 'port port)) > ] > (place-channel-put/get ch msg) > )) > : > ; other response/output stuff > : > ) > ) > > and while doing most processing in the worker place. Perhaps sanity > checking in the dispatch place before delegating. > > But I'm not positive this approach would work. Is that the actual TCP > port in response/output, or a local bytestring that will buffer the data in > the dispatch place before sending it on to the client? [Have to model that > and find out]. And dealing with a whole new set of potential errors due to > using places could get interesting. > > I can't afford to have data being copied/buffered unnecessarily. RAM isn't > an issue, but processing performance is. No point to relaying something > from place to place only to have both collectors have to recycle it later. > > The biggest issue I'm facing is that there's no practical way to limit the > data. The primary function of the application is to be a specialized DBMS > search engine. Most realistic queries are expected to produce < 250KB ... > but poorly targeted queries combined with [not my choice] an option to > "show all results" (as opposed to in "pages") have potential to produce > many Mbytes. Meanwhile the database - and with it the largest possible > result - keeps growing. The user base also is expected to grow > significantly in the next year. > > > Most of the time, the single core application handles its load just fine. > I have it limited to 16 concurrent requests and most searches return in a > few seconds with the odd large one absorbed nicely even when traffic is > saturating. But 99.9994% of all requests are some kind of search - every > other function combined lurks in the rounding error. Just a handful of > concurrent large queries can bog down the server for minutes while the > application struggles with post-processing and packaging results for the > clients. Apache and the DBMS are apportioned carefully over the available > cores - the limitation to more users and/or bigger searches on the current > hardware now is the application. > > So my application will be going "places" :-) ... it's only a matter of > exactly how. > > > George > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Jay McCarthy Associate Professor PLT @ CS @ UMass Lowell http://jeapostrophe.github.io "Wherefore, be not weary in well-doing, for ye are laying the foundation of a great work. And out of small things proceedeth that which is great." - D&C 64:33 -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+
[racket-users] Help with a bewildering bug.
I've been working on a new feature for Heresy and I've run into a bug I cannot even begin to track down the actual cause of. There's a Github issue here: https://github.com/jarcane/heresy/issues/37 I've tried a few naive solutions, but to be honest I really don't even know where to start because even the errors I get aren't even very informative. Run locally in the monadish module the code returns an error with line number and highlights the for loop in :>, run in another file I've had errors flag on both #%app and make-thing in the mentioned modules in the Github issue. Any pointers would be rather appreciated, I suspect there's some simple abuse of scope or hygiene I'm missing but I'm having trouble following the layers and layers of abstraction involved here. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
