[racket-users] Struct declaration conflict if a file is required implicitly

[Alejandro Sanchez]

Hello everyone,

I am working on this project: https://gitlab.com/HiPhish/MsgPack.rkt/

I am writing test cases and I ran into a problem with my ‘ext’ structure. It is 
declared in the file ‘msgpack/main.rkt’, which is required in the file 
‘msgpack/pack.rkt’ (also in ‘msgpack/unpack.rkt’). For my test case the test 
file looks like this:

(require
  quickcheck
  rackunit/quickcheck
  (file "../../msgpack/main.rkt")
  (file "../../msgpack/pack.rkt"))

(check-property
  (property ()
(let ([obj (ext #x01 (bytes #x02))])
  (bytes=? (call-with-output-bytes (λ (out) (pack obj out)))
   (bytes-append (bytes #xD4 (ext-type obj))
 (ext-data obj))

Here is what happens from my understanding: when I required ‘main.rkt’ the 
structure declaration got evaluated, creating all the functions that go along 
with it, including ‘ext’ and ‘ext?’. Then when I required the ‘pack.rkt’ file 
those declarations got evaluated again and created a new set of ext-related 
functions that just happen to have the same name. This is why I  the object I’m 
trying to pack falls through all the ‘cont’ cases.

This isn’t limited to the test file, I also tried the following on the REPL 
with the same results:

Welcome to Racket v6.9.
> (require msgpack msgpack/pack)
> (define e (ext 1 (bytes 2 3)))
> (define out (open-output-bytes))
> (pack e out)
; Type not supported by MessagePack [,bt for context]
> (pack-ext e out)
; pack-ext: contract violation
;   expected: ext?
;   given: #
;   in: the 1st argument of
;   (->
;ext?
;(and/c output-port? (not/c port-closed?))
;any)
;   contract from:
;   /msgpack-rkt/msgpack/pack.rkt
;   blaming: top-level
;(assuming the contract is correct)
;   at: /msgpack-rkt/msgpack/pack.rkt:53.5
; [,bt for context]
> ,bt
; pack-ext: contract violation
;   expected: ext?
;   given: #
;   in: the 1st argument of
;   (->
;ext?
;(and/c output-port? (not/c port-closed?))
;any)
;   contract from:
;   /msgpack-rkt/msgpack/pack.rkt
;   blaming: top-level
;(assuming the contract is correct)
;   at: /msgpack-rkt/msgpack/pack.rkt:53.5
;   context...:
;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/contract/private/blame.rkt:159:0:
 raise-blame-error16
;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/contract/private/arrow-val-first.rkt:357:18
;
/usr/local/Cellar/minimal-racket/6.9/share/racket/pkgs/xrepl-lib/xrepl/xrepl.rkt:1448:0
;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/private/misc.rkt:88:7
>

Adding a ‘#:prefab’ to the end of the struct declaration does not solve the 
problem. What am I doing wrong? Both the packing and unpacking need the ‘ext’ 
type for conformance with MessagePack. Do I have to make a large umbrella 
module that provides the entire API? I would prefer is users could just 
‘require’ parts of the library as they need them (i.e. only ‘(require 
msgpack/pack)’ if you only want to unpack data).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] Struct declaration conflict if a file is required implicitly

[Alejandro Sanchez]

Thank you so much! I feel so stupid now, that file path is a leftover from when 
the directory structure was different. Now it works perfectly.

> On 23 Jul 2017, at 17:43, Ryan Culpepper  wrote:
> 
> On 07/23/2017 07:26 AM, Alejandro Sanchez wrote:
>> Hello everyone,
>> I am working on this project: https://gitlab.com/HiPhish/MsgPack.rkt/
>> I am writing test cases and I ran into a problem with my ‘ext’ structure. It 
>> is declared in the file ‘msgpack/main.rkt’, which is required in the file 
>> ‘msgpack/pack.rkt’ (also in ‘msgpack/unpack.rkt’). For my test case the test 
>> file looks like this:
> 
> It looks like msgpack/pack.rkt requires "../main.rkt" rather than "main.rkt". 
> There isn't a "../main.rkt" checked in, so maybe you have a stale file 
> getting loaded? (It could be at "../main.rkt" or possibly 
> "../compiled/main_rkt.zo".)
> 
> Ryan

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

Hello dear Racketeers,

I have been writing an implementation of the MessagePack protocol for Racket
and I think the library is ready to be showcased now:

https://gitlab.com/HiPhish/MsgPack.rkt 


### What is MessagePack? ###

MessagePack is a binary data serialisation format. The website describes it
"like JSON but fast and small". Unlike JSON the goal is not a format that's
human-readable, but one that can be very quickly serialised, transported and
serialised.

http://msgpack.org/ 


### About the Racket implementation ###

My goal was to keep everything as simple as possible: there are only two
functions: pack and unpack. If there is more than one way of packing an object
the smallest format is selected automatically. Here is a taste:

   (require msgpack/pack msgpack/unpack)
   ;;; A wild hodgepodge to pack
   (define vec #(1 2 "hello" '(3 4) '() #t))
   ;;; A byte string of packed data
   (define packed
 (call-with-output-bytes (λ (out) (pack vec out
   ;;; Unpack the data again
   (define upacked
 (call-with-input-bytes packed (λ (in) (unpack in


As you can see, data is packed to and unpacked from a binary port. I think this
is better than packing/unpacking to binary string because MessagePack is
primarily used for inter-process communication, so there is not much point in
keeping the packed data inside a process.

I'd appreciate it a lot if a seasoned Racketeer could take a look at my code,
in particular if the library is set up properly (the info.rkt files), this is
my first time doing something in Racket. I am also open to suggestions about
the API, I haven't committed to version 1.0 yet. In particular, I am not
familiar with the modularity conventions of Racket libraries, i.e. if it is OK
to have 'msgpack/pack' and 'msgpack/unpack' or if everything should be covered
by one large 'provide' from 'msgpack'? There is one new type 'ext' declared,
should that be part of 'msgpack' or should I move it to 'msgpack/types'
instead?

On a related note, I find it really annoying that 'integer->integer-bytes' and
'integer-bytes->integer' do not support 8-bit integers. Is there a reason for
that? I had to write all sorts of ugly extra code for the 8-bit cases. I opened
an issue on GitHub about it (#1754).


### What's next? ###

Once the API settles I would like to move the library to typed Racket. I would
also like to submit it to the Racket packages catalog. The reason I wrote this
library is because I want to eventually write a Racket API client for Neovim:

https://github.com/neovim/neovim 
https://github.com/neovim/neovim/wiki/Related-projects#api-clients 


Neovim is a fork of Vim which aims to stay backwards compatible with Vim, but
at the same time bring the code base to modern standards, add long-wanted
features and make the editor easier to extend. They have already done a lot of
work, such asynchronous job control, a built-in terminal emulator, Lua
scripting and in particular a remote API.

The remote API allows one to write plugins in any language, provided there is a
client for that language. In contrast, Vim has to be compiled with support for
additional scripting languages and the integration burden was on the Vim
developers. This meant that popular languages like Python would be pretty well
supported, but more obscure languages were practically useless because no one
would re-compile their Vim just for one plugin. The remote API approach means
that Racket integration can be de-coupled from the editor development, and we
can write plugins that can make use of Racket libraries. One could for example
implement some of the DrRacket features using DrRacket as a library instead of
re-inventing the wheel. It would also be possible to integrate Neovim inside
DrRacket or write a Neovim GUI in Racket (GUIs are just very complex plugins in
Neovim).

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

> - I think you should have one module, `msgpack` that exports everything
OK. I my defense, I was originally planning to have a number of ‘pack-*’ 
methods like ‘pack-uin8’, ‘pack-uint16’ and so on, but there was nothing to be 
gained but a bloated interface.

> - You should rewrite your modules to use the `#lang racket/base`
> language so they don't force users to import so much stuff
Good idea, I’ll do that.

> - On integer->integer-bytes, I think that we should support 8-bit
> integers, but in the meantime, I think you should write your own
> version (integer->integer-bytes*) rather than having the handling in
> many places in the code.
Yes, I’ll do that eventually, I was just exhausted from getting it working and 
presentable first.

> - I'm curious of the performance. In particular, I would expect that a
> computed jump in unpack could do you good. Did you try that?
I haven’t investigated performance yet. As I said, I am new to Racket, this is 
my first time doing anything useful in it, my only previous Scheme knowledge 
was from doing the exercises in SICP and dabbling in Guile a bit. What is a 
computed jump?

> - Your package collection is 'multi, which is fine, but normally you
> just do that when you're defining something like data/heap or
> net/turkeyrpc, where you are extending some existing collection. In
> particular, you define msgpack and then you also define the test/pack
> collection (where you might expect it to be tests/msgpack/pack). I
> recommend having your collection be "msgpack" and putting your tests
> inside a tests sub-directory.
Just to make sure I understood correctly: ‘msgpack’ is the umbrella module that 
users import, ‘msgpack/test/pack’ (and ‘unpack’) are the test modules that will 
be run for testing only. How about the directory structure? I like to keep all 
source files in a source directory (my original reason for doing ‘multi), can I 
still do something like this?

|-README
|-LICENSE
|-info.rkt
|-source
  |-msgpack.rkt
  |-pack.rkt
  |-unpack.rkt
|-test
  |-pack.rkt
  |-pack
|- ...
  |-unpack.rkt
  |-unpack
|- …

It doesn’t have to be exactly this structure, but the idea is that all 
project-realted files are in the root, all the source files in the source 
directory and all the test files in the test directory.

> - On a style level, I think you should remove your lets and turn your
> if/begin blocks into conds, for example:
Good point.

> 
> On Mon, Jul 24, 2017 at 9:17 AM, Alejandro Sanchez
>  wrote:
>> Hello dear Racketeers,
>> 
>> I have been writing an implementation of the MessagePack protocol for Racket
>> and I think the library is ready to be showcased now:
>> 
>> https://gitlab.com/HiPhish/MsgPack.rkt
>> 
>> 
>> ### What is MessagePack? ###
>> 
>> MessagePack is a binary data serialisation format. The website describes it
>> "like JSON but fast and small". Unlike JSON the goal is not a format that's
>> human-readable, but one that can be very quickly serialised, transported and
>> serialised.
>> 
>> http://msgpack.org/
>> 
>> 
>> ### About the Racket implementation ###
>> 
>> My goal was to keep everything as simple as possible: there are only two
>> functions: pack and unpack. If there is more than one way of packing an
>> object
>> the smallest format is selected automatically. Here is a taste:
>> 
>>   (require msgpack/pack msgpack/unpack)
>>   ;;; A wild hodgepodge to pack
>>   (define vec #(1 2 "hello" '(3 4) '() #t))
>>   ;;; A byte string of packed data
>>   (define packed
>> (call-with-output-bytes (λ (out) (pack vec out
>>   ;;; Unpack the data again
>>   (define upacked
>> (call-with-input-bytes packed (λ (in) (unpack in
>> 
>> 
>> As you can see, data is packed to and unpacked from a binary port. I think
>> this
>> is better than packing/unpacking to binary string because MessagePack is
>> primarily used for inter-process communication, so there is not much point
>> in
>> keeping the packed data inside a process.
>> 
>> I'd appreciate it a lot if a seasoned Racketeer could take a look at my
>> code,
>> in particular if the library is set up properly (the info.rkt files), this
>> is
>> my first time doing something in Racket. I am also open to suggestions about
>> the API, I haven't committed to version 1.0 yet. In particular, I am not
>> familiar with the modularity conventions of Racket libraries, i.e. if it is
>> OK
>> to have 'msgpack/pack' and 'msgpack/unpack' or if everything should b

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

I will look into that later, one thing at a time :)

> On 24 Jul 2017, at 21:45, Vincent St-Amour  
> wrote:
> 
> Hi Alejandro,
> 
> This looks cool!
> 
> I don't see it listed at pkgs.racket-lang.org. It would be easier for
> users to discover it if you posted it there.
> 
> Vincent
> 
> 
> 
> On Mon, 24 Jul 2017 08:17:30 -0500,
> Alejandro Sanchez wrote:
>> 
>> Hello dear Racketeers,
>> 
>> I have been writing an implementation of the MessagePack protocol for Racket
>> and I think the library is ready to be showcased now:
>> 
>> https://gitlab.com/HiPhish/MsgPack.rkt
>> 
>> ### What is MessagePack? ###
>> 
>> MessagePack is a binary data serialisation format. The website describes it
>> "like JSON but fast and small". Unlike JSON the goal is not a format that's
>> human-readable, but one that can be very quickly serialised, transported and
>> serialised.
>> 
>> http://msgpack.org/
>> 
>> ### About the Racket implementation ###
>> 
>> My goal was to keep everything as simple as possible: there are only two
>> functions: pack and unpack. If there is more than one way of packing an 
>> object
>> the smallest format is selected automatically. Here is a taste:
>> 
>> (require msgpack/pack msgpack/unpack)
>> ;;; A wild hodgepodge to pack
>> (define vec #(1 2 "hello" '(3 4) '() #t))
>> ;;; A byte string of packed data
>> (define packed
>> (call-with-output-bytes (λ (out) (pack vec out
>> ;;; Unpack the data again
>> (define upacked
>> (call-with-input-bytes packed (λ (in) (unpack in
>> 
>> As you can see, data is packed to and unpacked from a binary port. I think 
>> this
>> is better than packing/unpacking to binary string because MessagePack is
>> primarily used for inter-process communication, so there is not much point in
>> keeping the packed data inside a process.
>> 
>> I'd appreciate it a lot if a seasoned Racketeer could take a look at my code,
>> in particular if the library is set up properly (the info.rkt files), this is
>> my first time doing something in Racket. I am also open to suggestions about
>> the API, I haven't committed to version 1.0 yet. In particular, I am not
>> familiar with the modularity conventions of Racket libraries, i.e. if it is 
>> OK
>> to have 'msgpack/pack' and 'msgpack/unpack' or if everything should be 
>> covered
>> by one large 'provide' from 'msgpack'? There is one new type 'ext' declared,
>> should that be part of 'msgpack' or should I move it to 'msgpack/types'
>> instead?
>> 
>> On a related note, I find it really annoying that 'integer->integer-bytes' 
>> and
>> 'integer-bytes->integer' do not support 8-bit integers. Is there a reason for
>> that? I had to write all sorts of ugly extra code for the 8-bit cases. I 
>> opened
>> an issue on GitHub about it (#1754).
>> 
>> ### What's next? ###
>> 
>> Once the API settles I would like to move the library to typed Racket. I 
>> would
>> also like to submit it to the Racket packages catalog. The reason I wrote 
>> this
>> library is because I want to eventually write a Racket API client for Neovim:
>> 
>> https://github.com/neovim/neovim
>> https://github.com/neovim/neovim/wiki/Related-projects#api-clients
>> 
>> Neovim is a fork of Vim which aims to stay backwards compatible with Vim, but
>> at the same time bring the code base to modern standards, add long-wanted
>> features and make the editor easier to extend. They have already done a lot 
>> of
>> work, such asynchronous job control, a built-in terminal emulator, Lua
>> scripting and in particular a remote API.
>> 
>> The remote API allows one to write plugins in any language, provided there 
>> is a
>> client for that language. In contrast, Vim has to be compiled with support 
>> for
>> additional scripting languages and the integration burden was on the Vim
>> developers. This meant that popular languages like Python would be pretty 
>> well
>> supported, but more obscure languages were practically useless because no one
>> would re-compile their Vim just for one plugin. The remote API approach means
>> that Racket integration can be de-coupled from the editor development, and we
>> can write plugins that can make use of Racket libraries. One could for 
>> example
>> implement some of the DrRacket features using DrRacket as a 

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

> On 24 Jul 2017, at 22:40, Jay McCarthy  wrote:
> 
> On Mon, Jul 24, 2017 at 3:18 PM, Alejandro Sanchez
>  wrote:
>>> - I'm curious of the performance. In particular, I would expect that a
>>> computed jump in unpack could do you good. Did you try that?
>> I haven’t investigated performance yet. As I said, I am new to Racket, this 
>> is my first time doing anything useful in it, my only previous Scheme 
>> knowledge was from doing the exercises in SICP and dabbling in Guile a bit. 
>> What is a computed jump?
> 
> Rather than having a big `cond`, you could look up the function that
> does the work in a vector and then call it. IMHO, msgpack was designed
> with that in mind, because tags that aren't immediate values are all
> nicely ordered. So you'd check the size, subtract a constant, and grab
> the appropriate procedure from a constant vector.
OK, that’s the sort of thing I would have done in C where the tag would be an 
index into an array of function pointers. Can you please point me to where in 
the manual it explains how to profile a single function?

>>> - Your package collection is 'multi, which is fine, but normally you
>>> just do that when you're defining something like data/heap or
>>> net/turkeyrpc, where you are extending some existing collection. In
>>> particular, you define msgpack and then you also define the test/pack
>>> collection (where you might expect it to be tests/msgpack/pack). I
>>> recommend having your collection be "msgpack" and putting your tests
>>> inside a tests sub-directory.
>> Just to make sure I understood correctly: ‘msgpack’ is the umbrella module 
>> that users import, ‘msgpack/test/pack’ (and ‘unpack’) are the test modules 
>> that will be run for testing only. How about the directory structure? I like 
>> to keep all source files in a source directory (my original reason for doing 
>> ‘multi), can I still do something like this?
>> 
>>|-README
>>|-LICENSE
>>|-info.rkt
>>|-source
>>  |-msgpack.rkt
>>  |-pack.rkt
>>  |-unpack.rkt
>>|-test
>>  |-pack.rkt
>>  |-pack
>>|- ...
>>  |-unpack.rkt
>>  |-unpack
>>|- …
>> 
>> It doesn’t have to be exactly this structure, but the idea is that all 
>> project-realted files are in the root, all the source files in the source 
>> directory and all the test files in the test directory.
> 
> You can do that, but you'd have to have an additional `main.rkt` file
> at the top-level that would require the things in `source` then
> re-export them. It is not really Racket style to do what you're
> talking about, however. If you did do that, then you could call the
> `source` directory, `private` and then it would have a Racket-y name,
> but your project isn't really large enough to warrant it and those
> files aren't actually provide.
> 
> Furthermore, your test/pack.rkt and test/unpack.rkt modules aren't
> necessary, because you should be testing with `raco test -c msgpack`,
> which will just go find everything. There's no need to build such
> things yourself. (Although, FWIW, I also wouldn't have separate those
> tests into such small files with just one or two, because they are,
> again, so small.)
I guess I’m weird that way, but I think of a project like a box. When you
buy a thing and open the box you want all the contents to be neatly
separated: here is the manual, here is the warranty card, here are the
parts, all wrapped nicely in a bag. You wouldn’t want the contents to be
loose and spill all over the floor. That’s why I like to separate the project
into directories by functionality (documentation, source, tests, manuals,
…). Oh well, if that is the Racket style I’ll do it your way.


>>> - On a style level, I think you should remove your lets and turn your
>>> if/begin blocks into conds, for example:
>> Good point.
>> 
>>> 
>>> On Mon, Jul 24, 2017 at 9:17 AM, Alejandro Sanchez
>>>  wrote:
>>>> Hello dear Racketeers,
>>>> 
>>>> I have been writing an implementation of the MessagePack protocol for 
>>>> Racket
>>>> and I think the library is ready to be showcased now:
>>>> 
>>>> https://gitlab.com/HiPhish/MsgPack.rkt
>>>> 
>>>> 
>>>> ### What is MessagePack? ###
>>>> 
>>>> MessagePack is a binary data serialisation format. The website describes it
>>>> "like JSON but fast and small". Unlike JSON the goal is not a format that's
>>>> hum

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

Thank you all for your help. I have moved to project to a flat file structure
now and reorganised everything hopefully to Racket standards (thank you,
Philip), and I have changed the complicated contracts to use 'integer-in'. I
think the API can stay as it is now, there is one struct and two functions, I
don't know anything that could be done that breaks backwards compatibility for
that. The package is up on the Racket package catalog:

https://pkgd.racket-lang.org/pkgn/package/msgpack

Do you guys think it is better to have a mandatory in/out port argument, or
should the port default to the current input/output port? I think the latter
makes it too easy to forget the port and I don't think users would want to use
stdin/stdout for this very often.

Another question: what about the name of the repository? Should it be renamed
to "msgpack" to match the name of the library or does the name not matter? I
can change the URL of the repository on GitLab, that would break all current
links, but at the moment that's not a big deal.

I'll clean the interior parts a bit more later (the 'unpack' dispatch and
cleaning the style), but that's not a priority as long as the external parts
are all set in place and working.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

Here is what I get from profiling the different dispatch methods in 'unpack'.
First I pack a positive fixnum which is among the first few conditions.

Cond:
cpu time: 810 real time: 3367 gc time: 0
cpu time: 815 real time: 3428 gc time: 0
cpu time: 815 real time: 3197 gc time: 0
cpu time: 806 real time: 3045 gc time: 0
cpu time: 806 real time: 3125 gc time: 0

cpu time: 811.5 real time: 3232.4

Case:
cpu time: 817 real time: 3122 gc time: 0
cpu time: 835 real time: 3544 gc time: 0
cpu time: 830 real time: 3337 gc time: 0
cpu time: 824 real time: 3216 gc time: 0
cpu time: 840 real time: 3667 gc time: 0

cpu time: 829.2 real time: 3377.2

Vector:
cpu time: 844 real time: 3839 gc time: 0
cpu time: 826 real time: 3253 gc time: 0
cpu time: 835 real time: 3194 gc time: 0
cpu time: 840 real time: 3165 gc time: 0
cpu time: 838 real time: 3354 gc time: 0

cpu time: 836.6 real time: 3361.0

Next I packed a negative fixnum, which is the last condition case:

Cond:
cpu time: 953 real time: 3556 gc time: 0
cpu time: 940 real time: 3035 gc time: 0
cpu time: 986 real time: 4250 gc time: 2
cpu time: 946 real time: 3425 gc time: 0
cpu time: 945 real time: 3233 gc time: 0

cpu time: 954.0 real time: 3499.8

Case:
cpu time: 890 real time: 2820 gc time: 0
cpu time: 896 real time: 2799 gc time: 0
cpu time: 904 real time: 2783 gc time: 2
cpu time: 884 real time: 2751 gc time: 0
cpu time: 887 real time: 2789 gc time: 0

cpu time: 892.2 real time: 2788.4

Vector:
cpu time: 866 real time: 2704 gc time: 0
cpu time: 884 real time: 2981 gc time: 0
cpu time: 880 real time: 2661 gc time: 3
cpu time: 864 real time: 2647 gc time: 0
cpu time: 864 real time: 2597 gc time: 0

cpu time: 871.6 real time: 2718.0

The average is below the line for each. I tested it by unpacking an array of
2^16 items, all the values were hard-coded so there is no chance of poor macros
messing things up. The dispatch vector is a vector of functions like this:

(define dispatch
  (vector
...
(lambda (in) (unpack-float 32 in))  ; #xCA
...
  ))

The unpacking function looks like this:

(define (unpack in)
  (define tag (read-byte in))
  (time ((vector-ref dispatch tag) in)))

So in conclusion it looks like the vector is the fastest when it comes to cases
later down the list while the 'cond' is the fastest at first, but slowest at
the end, slower than the vector at the beginning. The case averages somewhere
between the two.

Should I pick 'case' so I get average performance consistently, or should I
pick the vector so I get the best performance in the worst case? I don't think
one can predict which type of data users will most likely want to unpack on
average.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] [ANN] MessagePack implementation for Racke

[Alejandro Sanchez]

On Tuesday, July 25, 2017 at 11:37:48 PM UTC+2, Lehi Toskin wrote:
> One thing I'm curious about is what things can you and can you not pack? In 
> the README it shows bytes being packed, which seems a little obvious, but 
> what about (transparent) structs? Hashes? Lists? I'm very interested in this 
> package... for science!

I am not packing bytes in the readme, I am packing an integer in hexadecimal
notation so one can see easier how the integer corresponds to the packed bytes:
#x1234 becomes #xCD #x12 #x34, the first byte is the type tag for unsigned
16-bit integers and the other two are the bytes of the integer.

Packing means turning a Racket object into a byte sequence that can be sent to
another process. The recipient might not know anything about Racket, but it
still needs to be able to figure out how  to put the byte sequence back 
together,
that's why there is a type tag.

In theory you can pack anything as long as the recipient knows how it was
packed. Vectors and lists get packed into MessagePack arrays, hash tables into
MessagePack maps. If you want to pack a custom type like a point struct
'(struct point (x y))' you have a number of options. You could use the
MessagePack ext type which allows you to assign a custom type tag, then you
have to tell the recipient that for example the magic number 13 means point. If
the struct is transparent and you can 'read' and 'write' it in Racket you could
also pack the point as a string, but then you somehow have to make sure the
recipient knows that the string is meant to be an object and not just some
arbitrary text.

I think it would be worthwhile to later add the ability to define your own
packing functions so that you could then just call '(pack my-point out)'.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[racket-users] Need help porting a library to Typed Racket

[Alejandro Sanchez]

Hello guys,

I have been porting my MessagePack library to Typed Racket over the last few
days and I am almost done, except for some minor things I would like to sort
out first, so I was hoping some experienced Racketeers could help me out. Here
is the repo, the branch is "typed":

https://gitlab.com/HiPhish/MsgPack.rkt/tree/typed

For one, when testing I used to use file paths instead of the library name, but
after switching to Typed Racket I get this error:

  bash-3.2$ raco test test/unpack/extension.rkt
  raco test: (submod "test/unpack/extension.rkt" test)
  link: namespace mismatch;
   reference to a module that is not available
reference phase: 0
referenced module: (submod "/Users/hiphish/Developer/msgpack/unpack.rkt" 
#%contract-defs)
referenced phase level: 0
reference in module: (submod 
"/Users/hiphish/Developer/msgpack/test/unpack/extension.rkt" test)
in: idB4.0
context...:
 (submod /Users/hiphish/Developer/msgpack/test/unpack/extension.rkt test): 
[running body]
 
/usr/local/Cellar/minimal-racket/6.9/share/racket/pkgs/compiler-lib/compiler/commands/test.rkt:177:16
  extension.rkt: raco test: test raised an exception

This was after changing the require for `msgpack` to `(file "../../main.rkt")`.
The same happens in other test files as well.

The other thing is, I need to make the `ext` structure `#:transparent` or else
I get this error:

  > (call-with-input-bytes packed (lambda (in) (unpack in)))
  > ,bt
  ; unpack: broke its own contract
  ;   any-wrap/c: Unable to protect opaque value passed as `Any`
  ;   value: #
  ;   in: the range of
  ;   (-> Input-Port Any)
  ;   contract from: /msgpack/unpack.rkt
  ;   blaming: /msgpack/unpack.rkt
  ;(assuming the contract is correct)
  ;   at: /msgpack/unpack.rkt:24.9
  ;   context...:
  ;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/contract/private/blame.rkt:159:0:
 raise-blame-error16
  ;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/contract/combinator.rkt:176:4
  ;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/contract/private/arrow-higher-order.rkt:145:21
  ;
/usr/local/Cellar/minimal-racket/6.9/share/racket/pkgs/xrepl-lib/xrepl/xrepl.rkt:1448:0
  ;
/usr/local/Cellar/minimal-racket/6.9/share/racket/collects/racket/private/misc.rkt:88:7
  >

If I try unpacking the data in a Typed Racket REPL it works fine, the issue
only happens in regular Racket.

The next question is not a problem but more about style. In the `unpack.rkt`
module I need to read bytes, and the result is of type `(U Byte EOF)`, so I
need to use `cond` to distinguish between whether the result is a byte or the
`#`, and I raise an exception when an `#` is encountered where a byte
is expected. Is my way of doing it the right way?

And finally, what about the documentation? Does it need special treatment to
account for Typed Racket or can I leave it as it is?

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.