Re: encode $encoding list ???

[William Michels via perl6-users]

Hi Todd, I'm trying to follow what you're doing (below in Terminal app on
MacOS):

~$ raku
Welcome to 퐑퐚퐤퐮퐝퐨™ v2021.06.
Implementing the 퐑퐚퐤퐮™ programming language v6.d.
Built on MoarVM version 2021.06.

To exit type 'exit' or '^D'
> print Buf.new(0x84, 0x73, 0x77, 0x84, 0x79).decode("utf8-c8") ~ "\n"
�x84sw�x84y
>

Not clear if this is what you expect. I've also run code from
https://docs.raku.org/language/unicode#UTF8-C8 and see something different
from what's posted there:

my $test-file = "/tmp/test";
> given open($test-file, :w, :bin) {
>   .write: Buf.new(ord('A'), 0xFA, ord('B'), 0xFB, 0xFC, ord('C'), 0xFD);
>   .close;
> }
>
> say slurp($test-file, enc => 'utf8-c8');
> # OUTPUT: «(65 250 66 251 252 67 253)»
>

The output I actually see is:

A�xFAB�xFB�xFCC�xFD

If I go into /tmp and look at the file created, it contains the following
single line:

AúBûüCý


HTH, Bill.


On Sun, Jun 19, 2022 at 8:42 PM ToddAndMargo via perl6-users <
perl6-users@perl.org> wrote:

>  >print Buf.new(0x84, 0x73, 0x77, 0x84, 0x79).decode("utf8-c8") ~ "\n"
>
> �x84sw�x84y
>

Re: encode $encoding list ???

[ToddAndMargo via perl6-users]

>print Buf.new(0x84, 0x73, 0x77, 0x84, 0x79).decode("utf8-c8") ~ "\n"

�x84sw�x84y

Re: encode $encoding list ???

[ToddAndMargo via perl6-users]

On 6/19/22 02:30, William Michels via perl6-users wrote:

On Sun, Jun 19, 2022 at 2:16 AM ToddAndMargo via perl6-users
 wrote:


On 6/19/22 02:14, ToddAndMargo via perl6-users wrote:



On Sun, Jun 19, 2022 at 12:55 AM ToddAndMargo via perl6-users
 wrote:


Hi All,

https://docs.raku.org/routine/encode

multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool()
:$translate-nl = False, :$strict)

shows "ascii" and "utf8" as possible values for
"$encoding".

I tool a guess and found out "utf16".

Where are the rest of the values  R

-T


On 6/19/22 01:55, William Michels via perl6-users wrote:
  > Hi Todd,
  >
  > It's great that you've delved into the "encode" docs.
  > However to answer your question, I think you want to look at the
  > "encoding" routine page:
  >
  > https://docs.raku.org/routine/encoding#class_IO::Handle
  >
  > Here's the list on that page:
  >
  > utf8
  > utf16
  > utf16le
  > utf16be
  > utf8-c8
  > iso-8859-1
  > windows-1251
  > windows-1252
  > windows-932
  > ascii
  >
  > HTH, Bill.
  >
  >


Yes that helps greatly.  Thank you!



It would probably kill them to put a link to that
on the first page.


That's a great idea, and probably should be done. In the meanwhile
maybe you can test out some of the different encoding names--and let
us on the mailing-list know that everything is working okay?
Thx --B.


They all seem to work

I did not come up with a specific example for each though.

[0] > my Str $str = "Þor is mighty";
Þor is mighty
[1] > say $str.encode("ascii", :replacement( 'Th') ).decode("ascii");
Thor is mighty
[1] > say $str.encode("utf8", :replacement( 'Th') ).decode("utf8");
Þor is mighty
[1] > say $str.encode("utf16", :replacement( 'Th') ).decode("utf16");
Þor is mighty
[1] > say $str.encode("utf16le", :replacement( 'Th') ).decode("utf16le");
Þor is mighty
[1] > say $str.encode("utf8-c8", :replacement( 'Th') ).decode("utf8-c8");
Þor is mighty
[1] > say $str.encode("iso-8859-1", :replacement( 'Th') 
).decode("iso-8859-1");

Þor is mighty
[1] > say $str.encode("windows-1251", :replacement( 'Th') 
).decode("windows-1251");

Юor is mighty
[1] > say $str.encode("windows-1252", :replacement( 'Th') 
).decode("windows-1252");

Þor is mighty
[1] > say $str.encode("windows-932", :replacement( 'Th') 
).decode("windows-932");

Thor is mighty
[1] > say $str.encode("ascii", :replacement( 'Th') ).decode("ascii");
Thor is mighty





--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

My How To on Buffers

[ToddAndMargo via perl6-users]

Hi All,

If you were wondering what I was up to, this is it.

This is originally in Open Document Test (.odt) format,
so I do not know how well it will translate to text.
If you want the original document, I will email it
to you.  It is a lot pretties and you can 
on the Table of Contents to jump to the subject
you want

Thank you for all the help!

-T


Raku: Buffers, How To:

Table of Contents
References: 1
Type Graph: 1
Types of buffers:   2
Simple creation:2
Presalt with 10 entries of the same thing:  2
Presalt with spec entries:  2
Presalt with swept entries: 2
Counting the number of elements in a buffer:2
Append Strings (letters) to a buffer:   3
Append numbers to a buffer: 3
Changing elements of a buffer:  3
Copy elements from one buffer to another:   3
copy cell 1 through cell 2 from $x to $y:   4
copy three cells starting at cell 1 from $x to $y:  4
Locate a sub-buffer inside a buffer:4
String to Buffer:   5
Buffer to String:   5
int/uint to Buf:5
Num to Buf: 5
Num (square root of three) to Buf:  5
40 digits without the decimal point of the square root of 3:5

References:
   https://docs.raku.org/type/Buf
   https://docs.raku.org/routine/append
   https://raku.github.io/Documentable/integration-test/type
   https://docs.raku.org/type/Buf#(Blob)_method_subbuf
   https://docs.raku.org/images/type-graph-Buf.svg
   https://docs.raku.org/routine/encode

Type Graph:
  


Types of buffers:
Note: number refers to the number of bits
Buf
buf8 (default)
buf16
buf32
buf64


Simple creation:
Note: buffer indexes start counting from zero
   >my Buf $x;
   (Buf)

   > my $b = buf8.new( 0xAB, 0xAC );
   Buf[uint8]:0x

   > my buf8 $c = buf8.new( 0xAB, 0xAC );
   Buf[uint8]:0x

   > my Buf $d = buf8.new( 0xAB, 0xAC );
   Buf[uint8]:0x


Presalt with 10 entries of the same thing:
   > my Buf $y = Buf.new( 0xFA xx 10);
   Buf:0x


Presalt with specific entries:
   > my Buf $z = Buf.new( 0xAF, 0xFF, 0x1F );
   Buf:0x


Presalt with swept entries:
   > my buf8 $e = buf8.new(0x5..0x8);
   Buf[uint8]:0x<05 06 07 08>

   > my buf8 $e = buf8.new(0x5A..0x5D);
   Buf[uint8]:0x<5A 5B 5C 5D>


Counting the number of elements in a buffer:
Note: buffer indexes start counting from zero
   > say $y.bytes
   14
   > say $y.elems
   14

Append Strings (letters) to a buffer:
   $y ~= "A".encode.Buf;
   Buf:0x


Append numbers to a buffer:
   > my buf8 $x = buf8.new( 0x41, 0x42, 0x43, 0x44 )
   Buf[unit8]:0x<41 42 43 44>

   > $x.append( 0xDD );
   Buf:0x<41 42 43 44 DD>

   > $x.append( 0xEE..0xF1 );
   Buf:0x<41 42 43 44 DD EE EF F0 F1>

   > $x.append( 0xA1, 0xA3, 0xA5 );
   Buf:0x<41 42 43 44 DD EE EF F0 F1 A1 A3 A5>

   $x ~= buf8.new( 0xBB, 0xBC );
   Buf:0x<41 42 43 44 DD EE EF F0 F1 A1 A3 A5 BB BC>

   > $x.push( 0xDE, 0xDF );
   Buf[uint8]:0x<41 42 43 44 DD EE EF F0 F1 A1 A3 A5 BB BC DE DF>


Changing elements of a buffer:
   > say $y[3].base(0x10)
   FA
   > $y[3] = 0xFB
   251
   > say $y[3].base(16)
   FB


Copy elements from one buffer to another:
Note: buffer indexes start counting from zero
use `subbuf`

copy cell 1 through cell 2 from $x to $y:
> my Buf $x=Buf.new(0x55, 0x66, 0x77, 0x78);
Buf:0x<55 66 77 78>

> my Buf $y=$x.subbuf( 1..2 );
Buf:0x<66 77>

copy three cells starting at cell 1 from $x to $y:
> my Buf $x=Buf.new(0x55, 0x66, 0x77, 0x78);
Buf:0x<55 66 77 78>

> my Buf $y=$x.subbuf( 1, 3 );
Buf:0x<66 77 78>


Locate a sub-buffer inside a buffer:
On 2/5/19 7:55 AM, Brad Gilbert wrote:
`index` is an NQP op, which means in this case that it is written
in C (assuming you are using MoarVM)

I rewrote it using knowledge of the internals to be a bit faster:

sub blob-index ( Blob:D $buffer, Blob:D $needle, UInt:D $init = 0 
--> Int ) {

use nqp;
my int $needle-width = $needle.elems;
my int $elems = $buffer.elems;

if $elems < $needle-width + $init {
fail 'needle is larger than the buffer'
}

my uint $from = $init;
my uint $to   = $from + $needle-width - 1;

loop ( ; $to < $elems ; ++$from,++$to ) {
return $from if $needle eq nqp::slice($buffer,$from,$to)
# eq is safe since they are two Blobs/Bufs
}
return Nil
}

It's faster mainly because it doesn't use an iterator.

String to Buffer:
   > Buf.new("abc".encode)
   Buf:0x<61 62 63>

   say "abc".encode.Buf;
   Buf:0x<61 62 63>

   > my Buf $x;
   (Buf)
   > $x = "AB".encode.Buf;
   Buf:0x<41 42>
   $x ~= "CD".encode.Buf
   Buf:0x<41 42 43 44>

Buffer to String:
   > say Buf.new(97,98,99).decode
   abc

int/uint to Buf:
   > Buf.new(1414.Str.encode)
   Buf:0x<31 34 31 34>

Num to Buf:
   > Buf.new(2.1414.Str.encode)
   Buf:0x<32 2E 31 34 31 34>

Num (square root of three) to Buf:
>  Buf.new(3.sqrt.Str.encode)
Buf:0x<31 2E 37 33 32 30 35 30 38 30 37 35 36 38 38 37 37 32>

40 digits 

Re: Hex string to buf question

[ToddAndMargo via perl6-users]

On 6/19/22 03:06, ToddAndMargo via perl6-users wrote:

On 6/19/22 00:33, ToddAndMargo via perl6-users wrote:

Hi All,

I have a string:

 > my Str $x = "1BB67AE85A";
1BB67AE85A

which has the hexadecimal values I want to
add to a buffer:

 > my buf8 $y = buf8.new($x.base(16));
No such method 'base' for invocant of type 'Str'.  Did you mean any of
these: 'Bag', 'Date', 'Hash', 'are', 'asec', 'hash', 'take'?
   in block  at  line 1

In other words, I want to set $y to
    0x1B 0xB6 0x7A 0xE8 0x5A

How do I do this?

Many thanks,
-T





The guys on the chat line helped me figure it
out.  Give me a few minutes to pretty it up
and report back.



Hexadecimal string to Buffer:

Quick example:
> my Str $x = '0x84 0x73 0x77 0x84 0x79 0x87 0x84 0x68 0x73'

> my buf8 $y = buf8.new($x.match(/( ** 2)/, 
:g)».Str».parse-base(16));

Buf[uint8]:0x<84 73 77 84 79 87 84 68 73>


More in depth:

> use BigRoot;
Nil

>  BigRoot.precision = 10;
10

> my $BigNum  = BigRoot.newton's-sqrt: 11
3.3166247904

> my Str $MyCypher  = sprintf $BigNum.base(16);
3.510E527FE

> $MyCypher ~~ s/ '.' //;
｢.｣

> $MyCypher ~~ s:g/ (..) /0x$0, /;
(｢3｣
 0 => ｢3｣ ｢51｣
 0 => ｢51｣ ｢0E｣
 0 => ｢0E｣ ｢52｣
 0 => ｢52｣ ｢7F｣
 0 => ｢7F｣)

> $MyCypher ~~ s/ (.*) $(Q[,]) /$0/;
｢0x3, 0x51, 0x0E, 0x52, 0x7F,｣
 0 => ｢0x3, 0x51, 0x0E, 0x52, 0x7F｣

> my buf8 $MyCypherBuf = buf8.new($MyCypher.match(/( ** 2)/, 
:g)».Str».parse-base(16));

Buf[uint8]:0x<51 0E 52 7F>




--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: Hex string to buf question

[ToddAndMargo via perl6-users]

On 6/19/22 00:33, ToddAndMargo via perl6-users wrote:

Hi All,

I have a string:

 > my Str $x = "1BB67AE85A";
1BB67AE85A

which has the hexadecimal values I want to
add to a buffer:

 > my buf8 $y = buf8.new($x.base(16));
No such method 'base' for invocant of type 'Str'.  Did you mean any of
these: 'Bag', 'Date', 'Hash', 'are', 'asec', 'hash', 'take'?
   in block  at  line 1

In other words, I want to set $y to
    0x1B 0xB6 0x7A 0xE8 0x5A

How do I do this?

Many thanks,
-T





The guys on the chat line helped me figure it
out.  Give me a few minutes to pretty it up
and report back.

Re: Hex string to buf question

[ToddAndMargo via perl6-users]

On 6/19/22 00:33, ToddAndMargo via perl6-users wrote:

Hi All,

I have a string:

 > my Str $x = "1BB67AE85A";
1BB67AE85A

which has the hexadecimal values I want to
add to a buffer:

 > my buf8 $y = buf8.new($x.base(16));
No such method 'base' for invocant of type 'Str'.  Did you mean any of
these: 'Bag', 'Date', 'Hash', 'are', 'asec', 'hash', 'take'?
   in block  at  line 1

In other words, I want to set $y to
    0x1B 0xB6 0x7A 0xE8 0x5A

How do I do this?

Many thanks,
-T





Won't let me push an integer on a buffer either:

> my buf8 $y;

> $a
-68

> $y.push($a.Buf)
No such method 'Buf' for invocant of type 'Int'
  in block  at  line 1




--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: encode $encoding list ???

[William Michels via perl6-users]

On Sun, Jun 19, 2022 at 2:16 AM ToddAndMargo via perl6-users
 wrote:
>
> On 6/19/22 02:14, ToddAndMargo via perl6-users wrote:
> >
> >> On Sun, Jun 19, 2022 at 12:55 AM ToddAndMargo via perl6-users
> >>  wrote:
> >>>
> >>> Hi All,
> >>>
> >>> https://docs.raku.org/routine/encode
> >>>
> >>> multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool()
> >>> :$translate-nl = False, :$strict)
> >>>
> >>> shows "ascii" and "utf8" as possible values for
> >>> "$encoding".
> >>>
> >>> I tool a guess and found out "utf16".
> >>>
> >>> Where are the rest of the values  R
> >>>
> >>> -T
> >
> > On 6/19/22 01:55, William Michels via perl6-users wrote:
> >  > Hi Todd,
> >  >
> >  > It's great that you've delved into the "encode" docs.
> >  > However to answer your question, I think you want to look at the
> >  > "encoding" routine page:
> >  >
> >  > https://docs.raku.org/routine/encoding#class_IO::Handle
> >  >
> >  > Here's the list on that page:
> >  >
> >  > utf8
> >  > utf16
> >  > utf16le
> >  > utf16be
> >  > utf8-c8
> >  > iso-8859-1
> >  > windows-1251
> >  > windows-1252
> >  > windows-932
> >  > ascii
> >  >
> >  > HTH, Bill.
> >  >
> >  >
> >
> >
> > Yes that helps greatly.  Thank you!
> >
>
> It would probably kill them to put a link to that
> on the first page.

That's a great idea, and probably should be done. In the meanwhile
maybe you can test out some of the different encoding names--and let
us on the mailing-list know that everything is working okay?
Thx --B.

Re: encode $encoding list ???

[ToddAndMargo via perl6-users]

On 6/19/22 02:14, ToddAndMargo via perl6-users wrote:



On Sun, Jun 19, 2022 at 12:55 AM ToddAndMargo via perl6-users
 wrote:


Hi All,

https://docs.raku.org/routine/encode

multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool()
:$translate-nl = False, :$strict)

shows "ascii" and "utf8" as possible values for
"$encoding".

I tool a guess and found out "utf16".

Where are the rest of the values  R

-T


On 6/19/22 01:55, William Michels via perl6-users wrote:
 > Hi Todd,
 >
 > It's great that you've delved into the "encode" docs.
 > However to answer your question, I think you want to look at the
 > "encoding" routine page:
 >
 > https://docs.raku.org/routine/encoding#class_IO::Handle
 >
 > Here's the list on that page:
 >
 > utf8
 > utf16
 > utf16le
 > utf16be
 > utf8-c8
 > iso-8859-1
 > windows-1251
 > windows-1252
 > windows-932
 > ascii
 >
 > HTH, Bill.
 >
 >


Yes that helps greatly.  Thank you!



It would probably kill them to put a link to that
on the first page.


--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: encode $encoding list ???

[ToddAndMargo via perl6-users]

On Sun, Jun 19, 2022 at 12:55 AM ToddAndMargo via perl6-users
 wrote:


Hi All,

https://docs.raku.org/routine/encode

multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool()
:$translate-nl = False, :$strict)

shows "ascii" and "utf8" as possible values for
"$encoding".

I tool a guess and found out "utf16".

Where are the rest of the values  R

-T


On 6/19/22 01:55, William Michels via perl6-users wrote:
> Hi Todd,
>
> It's great that you've delved into the "encode" docs.
> However to answer your question, I think you want to look at the
> "encoding" routine page:
>
> https://docs.raku.org/routine/encoding#class_IO::Handle
>
> Here's the list on that page:
>
> utf8
> utf16
> utf16le
> utf16be
> utf8-c8
> iso-8859-1
> windows-1251
> windows-1252
> windows-932
> ascii
>
> HTH, Bill.
>
>


Yes that helps greatly.  Thank you!

--
~~
Computers are like air conditioners.
They malfunction when you open windows
~~

Re: encode $encoding list ???

[William Michels via perl6-users]

Hi Todd,

It's great that you've delved into the "encode" docs.
However to answer your question, I think you want to look at the
"encoding" routine page:

https://docs.raku.org/routine/encoding#class_IO::Handle

Here's the list on that page:

utf8
utf16
utf16le
utf16be
utf8-c8
iso-8859-1
windows-1251
windows-1252
windows-932
ascii

HTH, Bill.


On Sun, Jun 19, 2022 at 12:55 AM ToddAndMargo via perl6-users
 wrote:
>
> Hi All,
>
> https://docs.raku.org/routine/encode
>
> multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool()
> :$translate-nl = False, :$strict)
>
> shows "ascii" and "utf8" as possible values for
> "$encoding".
>
> I tool a guess and found out "utf16".
>
> Where are the rest of the values  R
>
> -T

encode $encoding list ???

[ToddAndMargo via perl6-users]

Hi All,

https://docs.raku.org/routine/encode

multi method encode(Str:D $encoding = 'utf8', :$replacement, Bool() 
:$translate-nl = False, :$strict)


shows "ascii" and "utf8" as possible values for
"$encoding".

I tool a guess and found out "utf16".

Where are the rest of the values  R

-T

Hex string to buf question

[ToddAndMargo via perl6-users]

Hi All,

I have a string:

> my Str $x = "1BB67AE85A";
1BB67AE85A

which has the hexadecimal values I want to
add to a buffer:

> my buf8 $y = buf8.new($x.base(16));
No such method 'base' for invocant of type 'Str'.  Did you mean any of
these: 'Bag', 'Date', 'Hash', 'are', 'asec', 'hash', 'take'?
  in block  at  line 1

In other words, I want to set $y to
   0x1B 0xB6 0x7A 0xE8 0x5A

How do I do this?

Many thanks,
-T


--

If I had a dime every time I didn't know
what was going on, I'd be like, "Why is
everyone giving me all these dimes?"

Re: append to Bug question

[ToddAndMargo via perl6-users]

Typo.

Should have been:

append to Buf question

And in my Keeper, I keep typing "bugger" instead
of "buffer".  Can't win.

Re: trouble returning a value from sub

[ToddAndMargo via perl6-users]

On 6/18/22 22:16, Bruce Gray wrote:




On Jun 18, 2022, at 10:42 PM, ToddAndMargo via perl6-users 
 wrote:

On 6/16/22 10:10, Rick Bychowski wrote:

sub MAIN($n = 20) {
.say for factors($n); # Nil
}



I thought `MAIN` was a reserved variable.  Am
I missing something?


MAIN has a special meaning as a sub name; it declares a CLI (command-line 
interface).

Just like this code declares a subroutine (that you can call within your 
program) that expects two filenames and an optional flag :
 sub do_it ( $file1, $file2, Bool :$dry-run ) {
 ...
 }
, the same signature in a sub named "MAIN" declares that your whole script is 
to be called on the command-line with two filenames and an optional flag :
 sub MAIN ( $file1, $file2, Bool :$dry-run ) {
 ...
 }

If I call that script like this:
 ./myscript.raku a.txt b.txt
, then MAIN gets 'a.txt' in $file1 and 'b.txt' in $file2. If I call it badly:
 ./myscript.raku a.txt b.txt c.txt
, then I get an error, with a usage message auto-generated by Raku:
 myscript.raku [--dry-run]  

For full details, see:
https://docs.raku.org/language/create-cli#index-entry-MAIN




I can definitely see a usage for that!  Thank you!


I currently use something like this:


# Note: @*ARGS[x] starts counting at 0
if @*ARGS.elems > 0  {
   $Usage   = lc "@*ARGS[0]";# lc = lower case
   if $Usage eq "--debug" || $Usage eq "debug" {
  $Debug = True;
  $Usage = "";
   } else {
  $RunSpecific = "@*ARGS[0]";
  $Debug   = True;
   }
}


if ( "$Usage" eq "--help" || "$Usage" eq "help" || "$Usage" eq "-?" || 
"$Usage" eq "/?" ) {

   PrintGreen "Usage: $IAm [ debug | --help ] [ Run_Specific_Module ] \n";
   PrintGreen "   $IAm debug Run_Specific_Module\n";
   PrintGreen "   $IAm debug GetADWCleaner\n\n";
   PrintGreen "Note: module name is case sensititive; debug 
and help are not\n";

   exit;
}

Re: append to Bug question

[ToddAndMargo via perl6-users]

On 6/18/22 22:58, Bruce Gray wrote:

If all you want is to append 0xBB to $y, either of these will work:
$y ~= Buf.new(0xBB);
$y.append(0xBB);



Did not realize I could use buf new like that.  Thank you!


Append numbers to a buffer:
   > $x
   Buf:0x<41 42 43 44>
   > $x.append( 0xDD );
   Buf:0x<41 42 43 44 DD>
   > $x.append( 0xEE..0xF1 );
   Buf:0x<41 42 43 44 DD EE EF F0 F1>
   > $x.append( 0xA1, 0xA3, 0xA5 );
   Buf:0x<41 42 43 44 DD EE EF F0 F1 A1 A3 A5>
   $x ~= buf8.new( 0xBB, 0xBC );
   Buf:0x<41 42 43 44 DD EE EF F0 F1 A1 A3 A5 BB BC>

Re: append to Bug question

[ToddAndMargo via perl6-users]

On 6/18/22 22:58, Bruce Gray wrote:


 From just looking at your code, I am not clear on what you are trying to do.


I am updating my Keeper on buffers.  I have four of them
that are a mess and I an going to consolidate them into
a single .ODT (open document text) file with an index.