Re: [Pharo-dev] Fun with AST evaluation

[Marcus Denker]

> On 18 Apr 2018, at 08:11, Marcus Denker  wrote:
> 
> 
> 
>> On 18 Apr 2018, at 07:41, Stephane Ducasse  wrote:
>> 
>> So this means that we could also have a simple hover showing the variable 
>> values when debugging. 
>> 
> It could be used for that, but then we did have already lookupSymbol: on 
> context (we added that so we
> can easily use any valid variable in the condition for conditional 
> breakpoints).
> 
Hmm… now that I think about it… I guess evaluateWithContext: could even be used 
to simplify the
code  that creates the condition block for conditional breakpoints.

This was not even the goal… to be checked.

Marcus

Re: [Pharo-dev] Fun with AST evaluation

[Marcus Denker]

> On 18 Apr 2018, at 07:41, Stephane Ducasse  wrote:
> 
> So this means that we could also have a simple hover showing the variable 
> values when debugging. 
> 
It could be used for that, but then we did have already lookupSymbol: on 
context (we added that so we
can easily use any valid variable in the condition for conditional breakpoints).

> 
> Now clement what is the difference with a plain AST interpreter?
> 

For larger ASTs it should be faster (especially with loops), for just reading a 
var the compilation overhead
should be quite high.

Did not measure it, though.

The nice thing that is is not a lot of code *and* it is the same code we use to 
evaluate DoIts already, so
with some refactoring is would be (from a LOC perspective) completely free and 
even simplify the Doit logic.

Marcus

Re: [Pharo-dev] pharo-contribution CI machines out of disk space

[Stephane Ducasse]

Yes we should. I started to keep the minimum on mine and remove some
obsolete ones.
The problem we have is that people forgot to remove their job when
they do not need them anymore.

Stef

On Wed, Apr 18, 2018 at 5:05 AM, monty  wrote:
> Not quite fixed: 
> https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1822/PHARO=30,VERSION=bleedingEdge,VM=vm/console
>
> Should we start reconfiguring jobs to keep fewer builds?
>
>> Sent: Tuesday, April 17, 2018 at 6:08 AM
>> From: "Christophe Demarey" 
>> To: "Pharo Development List" 
>> Subject: Re: [Pharo-dev] pharo-contribution CI machines out of disk space
>>
>>
>> > Le 17 avr. 2018 à 06:02, monty  a écrit :
>> >
>> > https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1820/console
>> >
>>
>> You’re right.
>> I did some cleaning and put slaves back on line.
>>
>> Thanks for the report.
>>
>

Re: [Pharo-dev] pharo-contribution CI machines out of disk space

[monty]

Not quite fixed: 
https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1822/PHARO=30,VERSION=bleedingEdge,VM=vm/console

Should we start reconfiguring jobs to keep fewer builds?

> Sent: Tuesday, April 17, 2018 at 6:08 AM
> From: "Christophe Demarey" 
> To: "Pharo Development List" 
> Subject: Re: [Pharo-dev] pharo-contribution CI machines out of disk space
>
> 
> > Le 17 avr. 2018 à 06:02, monty  a écrit :
> > 
> > https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1820/console
> > 
> 
> You’re right.
> I did some cleaning and put slaves back on line.
> 
> Thanks for the report.
>

[Pharo-dev] [Pharo 7.0-dev] Build #777: 21723-Opal-compiler-instances-are-created-in-huge-amount-during-a-method-compilation-from-the-browser

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #777 was: FAILURE.

The Pull Request #1212 was integrated: 
"21723-Opal-compiler-instances-are-created-in-huge-amount-during-a-method-compilation-from-the-browser"
Pull request url: https://github.com/pharo-project/pharo/pull/1212

Issue Url: https://pharo.fogbugz.com/f/cases/21723
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/777/

[Pharo-dev] [Pharo 7.0-dev] Build #776: 21722-Small-Cleanup-Syntax-highlighting-remove-a-reference-to-SHParserST80

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #776 was: SUCCESS.

The Pull Request #1210 was integrated: 
"21722-Small-Cleanup-Syntax-highlighting-remove-a-reference-to-SHParserST80"
Pull request url: https://github.com/pharo-project/pharo/pull/1210

Issue Url: https://pharo.fogbugz.com/f/cases/21722
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/776/

Re: [Pharo-dev] Pavel's ChangeLog week of 2018-02-05

[Thierry Goubier]

2018-04-17 14:28 GMT+02:00 Clément Bera :
>
>
> On Tue, Apr 17, 2018 at 2:08 PM, Thierry Goubier 
> wrote:
>>
>> 2018-04-17 14:03 GMT+02:00 Clément Bera :
>> > Hi Pavel,
>> >
>> > I'm looking at PosixSharedMemory again since I have to write a Master
>> > student proposal and I think this could be a good topic. I'm not really
>> > expert on SharedMemory, so I'm going to share what I have in mind, we
>> > can
>> > discuss about it, you could also co-supervise the student indirectly
>> > (though
>> > officially the supervisor has to be VUB staff).
>> >
>> > Implementation-wise, I was thinking what could be interesting are:
>> > - Making PosixSharedMemory compatible with TaskIt so from you image you
>> > can
>> > create a SharedMemory buffer, spawn another image+VM and attach it to
>> > the
>> > shared memory to have multiple threads working on the shared section.
>> > - Add the #at:if:put: primitive, which write into the shared memory
>> > using
>> > compare and swap instruction for efficient thread-safe access.
>> > - Add on SharedMemory all the primitives to read/write native types to
>> > the
>> > buffer (int64, double, etc.) with CAS and non CAS instructions.
>> > - Maybe add APIs to read/write objects through Fuel to pass them by
>> > copy,
>> > though this looks difficult in some cases.
>> > - Implement a lock system or a semaphore system on top of the CAS
>> > - implement lock-free and lock-full algorithm using CAS and non CAS
>> > instructions (I think a first try would be parallelSort on a 1Gb buffer
>> > of
>> > int32 with 4 native threads)
>> >
>> > What do you think ? Do you have ideas ? Are you interested ? Do you
>> > thing
>> > having a student on this would be nice ?
>> >
>> > The master thesis proposal has to include a research question. I am not
>> > sure
>> > what other languages do regarding shared memory.It's not clear so far
>> > what
>> > the research question is.
>>
>> I can forward that to a researcher here working on distributed shared
>> memory.
>>
>> Research question can be:
>> - heterogeneity (x86 + pi at the same time)
>> - load balancing between competing images on heterogeneous hardware
>> ...
>> - Object migration (pointer forwarding). Probably not, state of the
>> art is very advanced on that which means a costly implementation to
>> reach parity.
>>
>
> SharedMemory on heterogenous hardware ? Do you mean you need to physically
> plug the memory into a Raspberry Pie and an x86 computer ? Or you mean
> exporting the RAM of one hardware as NFS to the others ? I was just thinking
> sharing the memory between multiple pairs of image+VM on the same machine to
> be able to run some multi-threaded algorithm on the shared buffer. I know
> it's not much but we need to start somewhere.

No, it's having a memory abstraction (memory chunks) handled by a
server on a host (x86 or ARM), sending them over MPI to clients (x86
or ARM), each client accessing it through the OS shared memory,
releasing it when done so that other tasks can work on it (distributed
pipeline).

> The student would have 6 months, including 1 month to write the thesis so it
> cannot be too heavy. Something that we can re-use with a minor research
> contribution would be nice.

Given the state of the art in the field (20 years + of shared memory +
distributed shared memory + distributed object store and migration
already done), the only easy one that I know of is heterogeneity.

And even there, doing something worthy of a paper is hard.

Thierry

>> Thierry
>>
>> >
>> > Best,
>> >
>> > On Mon, Feb 12, 2018 at 10:33 AM, Pavel Krivanek
>> > 
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> among other less interesting things, I spent some time on existing
>> >> PosixSharedMemory project. It is a UFFI binding for the LibC methods
>> >> that provide support for the memory allocation between several
>> >> separate processes. I significantly improved the performance by
>> >> implementing the block access. Writing of 10MB byte array takes about
>> >> 1 millisecond, reading of it from other image took me about 4
>> >> milliseconds. While serialization with Fuel is very fast, it opens
>> >> interesting possibilities.
>> >> To have a shared memory without synchronization tools is not very
>> >> useful so I wrote a basic UFFI interface for the POSIX named
>> >> semaphores. They are quite easy to use and work nicely with Pharo. The
>> >> VM can all wait on the semaphore or it can check the status of it
>> >> periodically in an image thread. It has two small disadvantages. It
>> >> requires to dynamically link the next library (pthread) and they must
>> >> be cleaned manually. I plan to look at System V alternative in future.
>> >> Now we should write a nice framework for inter-image communication on
>> >> top of it or/and adopt Seamless for it ;-)
>> >>
>> >> Cheers,
>> >> -- Pavel
>> >>
>> >
>> >
>> >
>> > --
>> > Clément Béra
>> > https://clementbera.github.io/
>> > https://clementbera.wordpress.com/
>>
>
>
>
> --
> Clément Béra
> https:

Re: [Pharo-dev] Pavel's ChangeLog week of 2018-02-05

[Clément Bera]

On Tue, Apr 17, 2018 at 2:08 PM, Thierry Goubier 
wrote:

> 2018-04-17 14:03 GMT+02:00 Clément Bera :
> > Hi Pavel,
> >
> > I'm looking at PosixSharedMemory again since I have to write a Master
> > student proposal and I think this could be a good topic. I'm not really
> > expert on SharedMemory, so I'm going to share what I have in mind, we can
> > discuss about it, you could also co-supervise the student indirectly
> (though
> > officially the supervisor has to be VUB staff).
> >
> > Implementation-wise, I was thinking what could be interesting are:
> > - Making PosixSharedMemory compatible with TaskIt so from you image you
> can
> > create a SharedMemory buffer, spawn another image+VM and attach it to the
> > shared memory to have multiple threads working on the shared section.
> > - Add the #at:if:put: primitive, which write into the shared memory using
> > compare and swap instruction for efficient thread-safe access.
> > - Add on SharedMemory all the primitives to read/write native types to
> the
> > buffer (int64, double, etc.) with CAS and non CAS instructions.
> > - Maybe add APIs to read/write objects through Fuel to pass them by copy,
> > though this looks difficult in some cases.
> > - Implement a lock system or a semaphore system on top of the CAS
> > - implement lock-free and lock-full algorithm using CAS and non CAS
> > instructions (I think a first try would be parallelSort on a 1Gb buffer
> of
> > int32 with 4 native threads)
> >
> > What do you think ? Do you have ideas ? Are you interested ? Do you thing
> > having a student on this would be nice ?
> >
> > The master thesis proposal has to include a research question. I am not
> sure
> > what other languages do regarding shared memory.It's not clear so far
> what
> > the research question is.
>
> I can forward that to a researcher here working on distributed shared
> memory.
>
> Research question can be:
> - heterogeneity (x86 + pi at the same time)
> - load balancing between competing images on heterogeneous hardware
> ...
> - Object migration (pointer forwarding). Probably not, state of the
> art is very advanced on that which means a costly implementation to
> reach parity.
>
>
SharedMemory on heterogenous hardware ? Do you mean you need to physically
plug the memory into a Raspberry Pie and an x86 computer ? Or you mean
exporting the RAM of one hardware as NFS to the others ? I was just
thinking sharing the memory between multiple pairs of image+VM on the same
machine to be able to run some multi-threaded algorithm on the shared
buffer. I know it's not much but we need to start somewhere.

The student would have 6 months, including 1 month to write the thesis so
it cannot be too heavy. Something that we can re-use with a minor research
contribution would be nice.

Thierry
>
> >
> > Best,
> >
> > On Mon, Feb 12, 2018 at 10:33 AM, Pavel Krivanek <
> pavel.kriva...@gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> among other less interesting things, I spent some time on existing
> >> PosixSharedMemory project. It is a UFFI binding for the LibC methods
> >> that provide support for the memory allocation between several
> >> separate processes. I significantly improved the performance by
> >> implementing the block access. Writing of 10MB byte array takes about
> >> 1 millisecond, reading of it from other image took me about 4
> >> milliseconds. While serialization with Fuel is very fast, it opens
> >> interesting possibilities.
> >> To have a shared memory without synchronization tools is not very
> >> useful so I wrote a basic UFFI interface for the POSIX named
> >> semaphores. They are quite easy to use and work nicely with Pharo. The
> >> VM can all wait on the semaphore or it can check the status of it
> >> periodically in an image thread. It has two small disadvantages. It
> >> requires to dynamically link the next library (pthread) and they must
> >> be cleaned manually. I plan to look at System V alternative in future.
> >> Now we should write a nice framework for inter-image communication on
> >> top of it or/and adopt Seamless for it ;-)
> >>
> >> Cheers,
> >> -- Pavel
> >>
> >
> >
> >
> > --
> > Clément Béra
> > https://clementbera.github.io/
> > https://clementbera.wordpress.com/
>
>


-- 
Clément Béra
https://clementbera.github.io/
https://clementbera.wordpress.com/

Re: [Pharo-dev] Pavel's ChangeLog week of 2018-02-05

[Thierry Goubier]

2018-04-17 14:03 GMT+02:00 Clément Bera :
> Hi Pavel,
>
> I'm looking at PosixSharedMemory again since I have to write a Master
> student proposal and I think this could be a good topic. I'm not really
> expert on SharedMemory, so I'm going to share what I have in mind, we can
> discuss about it, you could also co-supervise the student indirectly (though
> officially the supervisor has to be VUB staff).
>
> Implementation-wise, I was thinking what could be interesting are:
> - Making PosixSharedMemory compatible with TaskIt so from you image you can
> create a SharedMemory buffer, spawn another image+VM and attach it to the
> shared memory to have multiple threads working on the shared section.
> - Add the #at:if:put: primitive, which write into the shared memory using
> compare and swap instruction for efficient thread-safe access.
> - Add on SharedMemory all the primitives to read/write native types to the
> buffer (int64, double, etc.) with CAS and non CAS instructions.
> - Maybe add APIs to read/write objects through Fuel to pass them by copy,
> though this looks difficult in some cases.
> - Implement a lock system or a semaphore system on top of the CAS
> - implement lock-free and lock-full algorithm using CAS and non CAS
> instructions (I think a first try would be parallelSort on a 1Gb buffer of
> int32 with 4 native threads)
>
> What do you think ? Do you have ideas ? Are you interested ? Do you thing
> having a student on this would be nice ?
>
> The master thesis proposal has to include a research question. I am not sure
> what other languages do regarding shared memory.It's not clear so far what
> the research question is.

I can forward that to a researcher here working on distributed shared memory.

Research question can be:
- heterogeneity (x86 + pi at the same time)
- load balancing between competing images on heterogeneous hardware
...
- Object migration (pointer forwarding). Probably not, state of the
art is very advanced on that which means a costly implementation to
reach parity.

Thierry

>
> Best,
>
> On Mon, Feb 12, 2018 at 10:33 AM, Pavel Krivanek 
> wrote:
>>
>> Hi,
>>
>> among other less interesting things, I spent some time on existing
>> PosixSharedMemory project. It is a UFFI binding for the LibC methods
>> that provide support for the memory allocation between several
>> separate processes. I significantly improved the performance by
>> implementing the block access. Writing of 10MB byte array takes about
>> 1 millisecond, reading of it from other image took me about 4
>> milliseconds. While serialization with Fuel is very fast, it opens
>> interesting possibilities.
>> To have a shared memory without synchronization tools is not very
>> useful so I wrote a basic UFFI interface for the POSIX named
>> semaphores. They are quite easy to use and work nicely with Pharo. The
>> VM can all wait on the semaphore or it can check the status of it
>> periodically in an image thread. It has two small disadvantages. It
>> requires to dynamically link the next library (pthread) and they must
>> be cleaned manually. I plan to look at System V alternative in future.
>> Now we should write a nice framework for inter-image communication on
>> top of it or/and adopt Seamless for it ;-)
>>
>> Cheers,
>> -- Pavel
>>
>
>
>
> --
> Clément Béra
> https://clementbera.github.io/
> https://clementbera.wordpress.com/

Re: [Pharo-dev] Pavel's ChangeLog week of 2018-02-05

[Clément Bera]

Hi Pavel,

I'm looking at PosixSharedMemory again since I have to write a Master
student proposal and I think this could be a good topic. I'm not really
expert on SharedMemory, so I'm going to share what I have in mind, we can
discuss about it, you could also co-supervise the student indirectly
(though officially the supervisor has to be VUB staff).

Implementation-wise, I was thinking what could be interesting are:
- Making PosixSharedMemory compatible with TaskIt so from you image you can
create a SharedMemory buffer, spawn another image+VM and attach it to the
shared memory to have multiple threads working on the shared section.
- Add the #at:if:put: primitive, which write into the shared memory using
compare and swap instruction for efficient thread-safe access.
- Add on SharedMemory all the primitives to read/write native types to the
buffer (int64, double, etc.) with CAS and non CAS instructions.
- Maybe add APIs to read/write objects through Fuel to pass them by copy,
though this looks difficult in some cases.
- Implement a lock system or a semaphore system on top of the CAS
- implement lock-free and lock-full algorithm using CAS and non CAS
instructions (I think a first try would be parallelSort on a 1Gb buffer of
int32 with 4 native threads)

What do you think ? Do you have ideas ? Are you interested ? Do you thing
having a student on this would be nice ?

The master thesis proposal has to include a research question. I am not
sure what other languages do regarding shared memory.It's not clear so far
what the research question is.

Best,

On Mon, Feb 12, 2018 at 10:33 AM, Pavel Krivanek 
wrote:

> Hi,
>
> among other less interesting things, I spent some time on existing
> PosixSharedMemory project. It is a UFFI binding for the LibC methods
> that provide support for the memory allocation between several
> separate processes. I significantly improved the performance by
> implementing the block access. Writing of 10MB byte array takes about
> 1 millisecond, reading of it from other image took me about 4
> milliseconds. While serialization with Fuel is very fast, it opens
> interesting possibilities.
> To have a shared memory without synchronization tools is not very
> useful so I wrote a basic UFFI interface for the POSIX named
> semaphores. They are quite easy to use and work nicely with Pharo. The
> VM can all wait on the semaphore or it can check the status of it
> periodically in an image thread. It has two small disadvantages. It
> requires to dynamically link the next library (pthread) and they must
> be cleaned manually. I plan to look at System V alternative in future.
> Now we should write a nice framework for inter-image communication on
> top of it or/and adopt Seamless for it ;-)
>
> Cheers,
> -- Pavel
>
>


-- 
Clément Béra
https://clementbera.github.io/
https://clementbera.wordpress.com/

Re: [Pharo-dev] [Pharo 7.0-dev] Build #775: Revert "21429-Watchpoints-should-be-cleaned-when-system-changed"

[Marcus Denker]

It is nice how easy a faulty merge can be reverted now… 

> On 17 Apr 2018, at 13:33, ci-pharo-ci-jenki...@inria.fr wrote:
> 
> There is a new Pharo build available!
>   
> The status of the build #775 was: SUCCESS.
> 
> The Pull Request #1209 was integrated: "Revert 
> "21429-Watchpoints-should-be-cleaned-when-system-changed""
> Pull request url: https://github.com/pharo-project/pharo/pull/1209
> 
> Issue Url: https://pharo.fogbugz.com/f/cases/revert
> Build Url: 
> https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/775/

[Pharo-dev] [Pharo 7.0-dev] Build #775: Revert "21429-Watchpoints-should-be-cleaned-when-system-changed"

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #775 was: SUCCESS.

The Pull Request #1209 was integrated: "Revert 
"21429-Watchpoints-should-be-cleaned-when-system-changed""
Pull request url: https://github.com/pharo-project/pharo/pull/1209

Issue Url: https://pharo.fogbugz.com/f/cases/revert
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/775/

[Pharo-dev] [Pharo 7.0-dev] Build #774: 21429-Watchpoints-should-be-cleaned-when-system-changed

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #774 was: FAILURE.

The Pull Request #1162 was integrated: 
"21429-Watchpoints-should-be-cleaned-when-system-changed"
Pull request url: https://github.com/pharo-project/pharo/pull/1162

Issue Url: https://pharo.fogbugz.com/f/cases/21429
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/774/

Re: [Pharo-dev] pharo-contribution CI machines out of disk space

[Christophe Demarey]

> Le 17 avr. 2018 à 06:02, monty  a écrit :
> 
> https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/1820/console
> 

You’re right.
I did some cleaning and put slaves back on line.

Thanks for the report.

[Pharo-dev] [Pharo 7.0-dev] Build #773: 21523-package-ReflectionMirrors-Primitives-needs-to-be-moved-to-the-BaselineOfPharoBootstrap

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #773 was: SUCCESS.

The Pull Request #1051 was integrated: 
"21523-package-ReflectionMirrors-Primitives-needs-to-be-moved-to-the-BaselineOfPharoBootstrap"
Pull request url: https://github.com/pharo-project/pharo/pull/1051

Issue Url: https://pharo.fogbugz.com/f/cases/21523
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/773/

Re: [Pharo-dev] [Pharo-Launcher] call for tests on Windows

[Christophe Demarey]

Windows 10 on top of virtual box

> Le 17 avr. 2018 à 10:29, p...@highoctane.be a écrit :
> 
> Hi Christophe,
> 
> What Windows version do you have?
> 
> I have Windows 10 and a 4K screen.
> 
> Phil
> 
> On Tue, Apr 17, 2018 at 9:43 AM, Christophe Demarey 
> mailto:christophe.dema...@inria.fr>> wrote:
> Hi Phil, Eliot,
> 
>> Le 16 avr. 2018 à 23:58, Eliot Miranda > > a écrit :
>> 
>> Hi Phil,
>> 
>> On Mon, Apr 16, 2018 at 12:54 PM, p...@highoctane.be 
>>  mailto:p...@highoctane.be>> 
>> wrote:
>> I have downloaded the 
>> https://github.com/pharo-project/pharo-vm/blob/master/opensmalltalk-vm/build.win32x86/pharo.cog.spur.lowcode/Pharo.exe.manifest
>>  
>> 
>>  and put it in the folder that contains the PharoLauncher Pharo.exe and then 
>> the pointers are correct again (they have their masking done correctly.
>> 
>> Good!  
>>  
>> So, there is a packaging issue somewhere for Pharo Windows VMs as downloaded 
>> by the Laucher.
>> 
>> Indeed, we must include the manifest files on Windows.
> 
> I did not get this problem. Is this manifest file new? Is there only one 
> manifest file to include or many?
> 
> Thanks for reporting this issue.
> Christophe
> 
> 

Re: [Pharo-dev] Environment variables encoding ?

[Henrik Sperre Johansen]

Damien Pollet wrote
> It seems macOS normalizes UTF-8 differently from everyone else in file
> names (I think base character + composing instead of precomposed
> codepoint). That might affect PWD.
> For environment variables, even if most sensible platforms should have
> adopted UTF-8 by now, I wouldn't be surprised if there's no official
> encoding whatsoever (i.e. they're just bytes with a 0 at the end…)
> 
> On 17 April 2018 at 09:36, Sven Van Caekenberghe <

> sven@

> > wrote:
> 
>> Hi,
>>
>> The dictionary
>>
>>  OSPlatform current environment
>>
>> contains a copy of the OS's environment variables (more correctly of the
>> VM process), as key/value pairs.
>>
>> These are obtained via the following system calls:
>>
>> on macOS & *nix
>>
>>   LIBC environ
>>
>> on Windows
>>
>>   KERNEL32 GetEnvironmentStrings
>>
>> It is however a bit unclear how these are encoded. On macOS & *nix that
>> seems to be UTF8, on Windows there are some reports that it appears to be
>> Latin1 - but both might be locale specific, I don't know either way.
>>
>> Does anyone know for sure ?
>>
>> I furthermore think that OSEnvironment and its subclasses, who do this
>> call, should be responsible for decoding the C strings into proper Pharo
>> strings, and not leave that responsibility to its users.
>>
>> Fundamentally, in the following, the decoding is still not done correctly
>> and that is wrong/confusing IMHO.
>>
>> $ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current
>> associations'
>> {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'.
>> 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/
>> sndrtj9j1tq06j0lfnshmrl8gn/T/'. 'FOO'->'benoît'.
>> 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
>> 'TERM_PROGRAM_VERSION'->'404'.
>> 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'.
>> 'USER'->'sven'.
>> 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'.
>> 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
>> 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'.
>> 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'.
>> 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com.
>> apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
>> 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'.
>> '_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'.
>> '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}
>>
>> Of course, if we change this, we will need to fix callers.
>>
>> Opinions ?
>>
>> Sven
>>
>> PS: Furthermore, I note that there is a subtle difference in how $FOO and
>> $PWD in the above are UTF-8 encoded. In the former, normalisation was
>> done,
>> in the latter not. Maybe that could lead to problems (when
>> comparing/composing them). This is a difficult/complex subject (
>> https://medium.com/concerning-pharo/an-implementation-of-unicode-
>> normalization-7c6719068f43).
>>
>>
>>
>>
> 
> 
> -- 
> Damien Pollet
> type less, do more [ | ] http://people.untyped.org/damien.pollet

If by different, you mean that it actually normalizes the file names, then
yes.
All Mac filenames are in a well defined form; NFD.
On linux, they're just arrays of bytes, and anything goes.
That the bytes mostly happen to be valid utf8 strings in NFC, is just a
by-product of the fact that's the format most programs use when calling the
file primitives. 

Cheers,
Henry



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Re: [Pharo-dev] Environment variables encoding ?

[Sven Van Caekenberghe]

Yes, I also saw the *W variants, my initial reaction was that those would make 
more sense, but it is hard to see all consequences.

> On 17 Apr 2018, at 10:49, Henrik Sperre Johansen 
>  wrote:
> 
> primitiveGetenv returns values in the current locale's code page on Windows; 
> a value bound to €  returns a stings with single char 128 on MS1252 (western
> european) at least.
> 
> On windows, there are three versions of each api call with string
> parameters/returns; 
> xxx (depending on UNICODE being defined, either resolves to *A or *W)
> xxxA (Ascii, or, more accurately, current code page)*
> xxxW (UTF-16)
> 
> IIRC, the intention is that primitives receiving/sending char* to the image
> will expect/return utf8, so a conversion macro before passing it on to the
> syscall would be necessary on Windows; I believe one exist already and is
> used in at least the file plugin primitives.
> 
> Cheers,
> Henry
> 
> * The windows FFI fallback used if primitive fails calls the *A version
> directly, and can be changed to call *W correctly, but there's a fair bit of
> wrapping fluff involved; 
> https://youtu.be/Um41DPPs5ZA?list=PL843D1D545F9F52B6&t=1591
> 
> 
> 
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
> 

Re: [Pharo-dev] Environment variables encoding ?

[Sven Van Caekenberghe]

> On 17 Apr 2018, at 10:40, Nicolai Hess  wrote:
> 
> 
> 
> 2018-04-17 10:05 GMT+02:00 Sven Van Caekenberghe :
> 
> 
> > On 17 Apr 2018, at 09:57, Damien Pollet  wrote:
> > 
> > It seems macOS normalizes UTF-8 differently from everyone else in file 
> > names (I think base character + composing instead of precomposed 
> > codepoint). That might affect PWD.
> > For environment variables, even if most sensible platforms should have 
> > adopted UTF-8 by now, I wouldn't be surprised if there's no official 
> > encoding whatsoever (i.e. they're just bytes with a 0 at the end…)
> 
> ;-)
> 
> We can decode everything, we have all the tools, but of course, we first have 
> to know what encoding is being used. Hence my question.
> 
> > On 17 April 2018 at 09:36, Sven Van Caekenberghe  wrote:
> > Hi,
> > 
> > The dictionary 
> > 
> >  OSPlatform current environment
> > 
> > contains a copy of the OS's environment variables (more correctly of the VM 
> > process), as key/value pairs. 
> > 
> > These are obtained via the following system calls:
> > 
> > on macOS & *nix
> > 
> >   LIBC environ
> > 
> > on Windows
> > 
> >   KERNEL32 GetEnvironmentStrings
> 
> 
> Interestingly, this is only for the dictionary operations (asDictionary, 
> keysAndValuesDo...)
> If you just access the variable with getEnv, it works:
> 
> OSPlatform current environment setEnv:'FOO' value:'benoît'.
> OSPlatform current environment getEnv:'FOO'. "'benoît'"
> OSPlatform current environment asDictionary at: 'FOO'. "'benoŒt'"

Hmm, not for me (on macOS):

$ FOO=benoît ./pharo Pharo.image eval "OSPlatform current environment 
getEnv:'FOO'"
'benoît'

If you put it in yourself, are you not cheating then ?

> > 
> > It is however a bit unclear how these are encoded. On macOS & *nix that 
> > seems to be UTF8, on Windows there are some reports that it appears to be 
> > Latin1 - but both might be locale specific, I don't know either way.
> > 
> > Does anyone know for sure ?
> > 
> > I furthermore think that OSEnvironment and its subclasses, who do this 
> > call, should be responsible for decoding the C strings into proper Pharo 
> > strings, and not leave that responsibility to its users.
> > 
> > Fundamentally, in the following, the decoding is still not done correctly 
> > and that is wrong/confusing IMHO.
> > 
> > $ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations' 
> > {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 
> > 'SHELL'->'/bin/bash'. 
> > 'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl8gn/T/'. 
> > 'FOO'->'benoît'. 
> > 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
> >  'TERM_PROGRAM_VERSION'->'404'. 
> > 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 
> > 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 
> > 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
> >  'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 
> > 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 
> > 'LC_CTYPE'->'UTF-8'. 
> > 'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
> >  'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'. 
> > '_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. 
> > '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}
> > 
> > Of course, if we change this, we will need to fix callers.
> > 
> > Opinions ?
> > 
> > Sven
> > 
> > PS: Furthermore, I note that there is a subtle difference in how $FOO and 
> > $PWD in the above are UTF-8 encoded. In the former, normalisation was done, 
> > in the latter not. Maybe that could lead to problems (when 
> > comparing/composing them). This is a difficult/complex subject 
> > (https://medium.com/concerning-pharo/an-implementation-of-unicode-normalization-7c6719068f43).
> > 
> > 
> > 
> > 
> > 
> > 
> > -- 
> > Damien Pollet
> > type less, do more [ | ] http://people.untyped.org/damien.pollet

Re: [Pharo-dev] Environment variables encoding ?

[Henrik Sperre Johansen]

primitiveGetenv returns values in the current locale's code page on Windows; 
a value bound to €  returns a stings with single char 128 on MS1252 (western
european) at least.

On windows, there are three versions of each api call with string
parameters/returns; 
xxx (depending on UNICODE being defined, either resolves to *A or *W)
xxxA (Ascii, or, more accurately, current code page)*
xxxW (UTF-16)

IIRC, the intention is that primitives receiving/sending char* to the image
will expect/return utf8, so a conversion macro before passing it on to the
syscall would be necessary on Windows; I believe one exist already and is
used in at least the file plugin primitives.

Cheers,
Henry

* The windows FFI fallback used if primitive fails calls the *A version
directly, and can be changed to call *W correctly, but there's a fair bit of
wrapping fluff involved; 
https://youtu.be/Um41DPPs5ZA?list=PL843D1D545F9F52B6&t=1591



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Re: [Pharo-dev] Environment variables encoding ?

[Nicolai Hess]

2018-04-17 10:05 GMT+02:00 Sven Van Caekenberghe :

>
>
> > On 17 Apr 2018, at 09:57, Damien Pollet  wrote:
> >
> > It seems macOS normalizes UTF-8 differently from everyone else in file
> names (I think base character + composing instead of precomposed
> codepoint). That might affect PWD.
> > For environment variables, even if most sensible platforms should have
> adopted UTF-8 by now, I wouldn't be surprised if there's no official
> encoding whatsoever (i.e. they're just bytes with a 0 at the end…)
>
> ;-)
>
> We can decode everything, we have all the tools, but of course, we first
> have to know what encoding is being used. Hence my question.
>
> > On 17 April 2018 at 09:36, Sven Van Caekenberghe  wrote:
> > Hi,
> >
> > The dictionary
> >
> >  OSPlatform current environment
> >
> > contains a copy of the OS's environment variables (more correctly of the
> VM process), as key/value pairs.
> >
> > These are obtained via the following system calls:
> >
> > on macOS & *nix
> >
> >   LIBC environ
> >
> > on Windows
> >
> >   KERNEL32 GetEnvironmentStrings
>


Interestingly, this is only for the dictionary operations (asDictionary,
keysAndValuesDo...)
If you just access the variable with getEnv, it works:

OSPlatform current environment setEnv:'FOO' value:'benoît'.
OSPlatform current environment getEnv:'FOO'. "'benoît'"
OSPlatform current environment asDictionary at: 'FOO'. "'benoŒt'"



> >
> > It is however a bit unclear how these are encoded. On macOS & *nix that
> seems to be UTF8, on Windows there are some reports that it appears to be
> Latin1 - but both might be locale specific, I don't know either way.
> >
> > Does anyone know for sure ?
> >
> > I furthermore think that OSEnvironment and its subclasses, who do this
> call, should be responsible for decoding the C strings into proper Pharo
> strings, and not leave that responsibility to its users.
> >
> > Fundamentally, in the following, the decoding is still not done
> correctly and that is wrong/confusing IMHO.
> >
> > $ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current
> associations'
> > {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'.
> 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/
> sndrtj9j1tq06j0lfnshmrl8gn/T/'. 'FOO'->'benoît'.
> 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
> 'TERM_PROGRAM_VERSION'->'404'. 
> 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'.
> 'USER'->'sven'. 
> 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'.
> 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
> 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'.
> 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'.
> 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com.
> apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
> 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'.
> '_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'.
> '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}
> >
> > Of course, if we change this, we will need to fix callers.
> >
> > Opinions ?
> >
> > Sven
> >
> > PS: Furthermore, I note that there is a subtle difference in how $FOO
> and $PWD in the above are UTF-8 encoded. In the former, normalisation was
> done, in the latter not. Maybe that could lead to problems (when
> comparing/composing them). This is a difficult/complex subject (
> https://medium.com/concerning-pharo/an-implementation-of-unicode-
> normalization-7c6719068f43).
> >
> >
> >
> >
> >
> >
> > --
> > Damien Pollet
> > type less, do more [ | ] http://people.untyped.org/damien.pollet
>
>
>

Re: [Pharo-dev] [Pharo-Launcher] call for tests on Windows

[p...@highoctane.be]

Hi Christophe,

What Windows version do you have?

I have Windows 10 and a 4K screen.

Phil

On Tue, Apr 17, 2018 at 9:43 AM, Christophe Demarey <
christophe.dema...@inria.fr> wrote:

> Hi Phil, Eliot,
>
> Le 16 avr. 2018 à 23:58, Eliot Miranda  a écrit :
>
> Hi Phil,
>
> On Mon, Apr 16, 2018 at 12:54 PM, p...@highoctane.be 
>  wrote:
>
>> I have downloaded the https://github.com/pharo-p
>> roject/pharo-vm/blob/master/opensmalltalk-vm/build.win32x86/
>> pharo.cog.spur.lowcode/Pharo.exe.manifest and put it in the folder that
>> contains the PharoLauncher Pharo.exe and then the pointers are correct
>> again (they have their masking done correctly.
>>
>
> Good!
>
>
>> So, there is a packaging issue somewhere for Pharo Windows VMs as
>> downloaded by the Laucher.
>>
>
> Indeed, we must include the manifest files on Windows.
>
>
> I did not get this problem. Is this manifest file new? Is there only one
> manifest file to include or many?
>
> Thanks for reporting this issue.
> Christophe
>
>

Re: [Pharo-dev] Environment variables encoding ?

[Sven Van Caekenberghe]

> On 17 Apr 2018, at 09:57, Damien Pollet  wrote:
> 
> It seems macOS normalizes UTF-8 differently from everyone else in file names 
> (I think base character + composing instead of precomposed codepoint). That 
> might affect PWD.
> For environment variables, even if most sensible platforms should have 
> adopted UTF-8 by now, I wouldn't be surprised if there's no official encoding 
> whatsoever (i.e. they're just bytes with a 0 at the end…)

;-)

We can decode everything, we have all the tools, but of course, we first have 
to know what encoding is being used. Hence my question.

> On 17 April 2018 at 09:36, Sven Van Caekenberghe  wrote:
> Hi,
> 
> The dictionary 
> 
>  OSPlatform current environment
> 
> contains a copy of the OS's environment variables (more correctly of the VM 
> process), as key/value pairs. 
> 
> These are obtained via the following system calls:
> 
> on macOS & *nix
> 
>   LIBC environ
> 
> on Windows
> 
>   KERNEL32 GetEnvironmentStrings
> 
> It is however a bit unclear how these are encoded. On macOS & *nix that seems 
> to be UTF8, on Windows there are some reports that it appears to be Latin1 - 
> but both might be locale specific, I don't know either way.
> 
> Does anyone know for sure ?
> 
> I furthermore think that OSEnvironment and its subclasses, who do this call, 
> should be responsible for decoding the C strings into proper Pharo strings, 
> and not leave that responsibility to its users.
> 
> Fundamentally, in the following, the decoding is still not done correctly and 
> that is wrong/confusing IMHO.
> 
> $ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations' 
> {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 
> 'SHELL'->'/bin/bash'. 
> 'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl8gn/T/'. 
> 'FOO'->'benoît'. 
> 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
>  'TERM_PROGRAM_VERSION'->'404'. 
> 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 
> 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 
> 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
>  'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 
> 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 
> 'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
>  'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'. 
> '_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. 
> '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}
> 
> Of course, if we change this, we will need to fix callers.
> 
> Opinions ?
> 
> Sven
> 
> PS: Furthermore, I note that there is a subtle difference in how $FOO and 
> $PWD in the above are UTF-8 encoded. In the former, normalisation was done, 
> in the latter not. Maybe that could lead to problems (when 
> comparing/composing them). This is a difficult/complex subject 
> (https://medium.com/concerning-pharo/an-implementation-of-unicode-normalization-7c6719068f43).
> 
> 
> 
> 
> 
> 
> -- 
> Damien Pollet
> type less, do more [ | ] http://people.untyped.org/damien.pollet

Re: [Pharo-dev] Environment variables encoding ?

[Damien Pollet]

It seems macOS normalizes UTF-8 differently from everyone else in file
names (I think base character + composing instead of precomposed
codepoint). That might affect PWD.
For environment variables, even if most sensible platforms should have
adopted UTF-8 by now, I wouldn't be surprised if there's no official
encoding whatsoever (i.e. they're just bytes with a 0 at the end…)

On 17 April 2018 at 09:36, Sven Van Caekenberghe  wrote:

> Hi,
>
> The dictionary
>
>  OSPlatform current environment
>
> contains a copy of the OS's environment variables (more correctly of the
> VM process), as key/value pairs.
>
> These are obtained via the following system calls:
>
> on macOS & *nix
>
>   LIBC environ
>
> on Windows
>
>   KERNEL32 GetEnvironmentStrings
>
> It is however a bit unclear how these are encoded. On macOS & *nix that
> seems to be UTF8, on Windows there are some reports that it appears to be
> Latin1 - but both might be locale specific, I don't know either way.
>
> Does anyone know for sure ?
>
> I furthermore think that OSEnvironment and its subclasses, who do this
> call, should be responsible for decoding the C strings into proper Pharo
> strings, and not leave that responsibility to its users.
>
> Fundamentally, in the following, the decoding is still not done correctly
> and that is wrong/confusing IMHO.
>
> $ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations'
> {'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'.
> 'SHELL'->'/bin/bash'. 'TMPDIR'->'/var/folders/sy/
> sndrtj9j1tq06j0lfnshmrl8gn/T/'. 'FOO'->'benoît'.
> 'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
> 'TERM_PROGRAM_VERSION'->'404'. 
> 'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'.
> 'USER'->'sven'. 
> 'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'.
> 'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
> 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'.
> 'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'.
> 'LC_CTYPE'->'UTF-8'. 'DISPLAY'->'/private/tmp/com.
> apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
> 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'.
> '_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'.
> '__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}
>
> Of course, if we change this, we will need to fix callers.
>
> Opinions ?
>
> Sven
>
> PS: Furthermore, I note that there is a subtle difference in how $FOO and
> $PWD in the above are UTF-8 encoded. In the former, normalisation was done,
> in the latter not. Maybe that could lead to problems (when
> comparing/composing them). This is a difficult/complex subject (
> https://medium.com/concerning-pharo/an-implementation-of-unicode-
> normalization-7c6719068f43).
>
>
>
>


-- 
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet

Re: [Pharo-dev] Environment variables encoding ?

[Benoit St-Jean via Pharo-dev]

--- Begin Message ---
I do remember clearly that while debugging that problem, the %LOCALAPPDATA% 
environment at some point kept that string encoded with Latin-1 (I'm on Windows 
10, french version).  I have not been able to reproduce the exact sequence 
which led to that specific case unfortunately...


- 
Benoît St-Jean 
Yahoo! Messenger: bstjean 
Twitter: @BenLeChialeux 
Pinterest: benoitstjean 
Instagram: Chef_Benito
IRC: lamneth 
Blogue: endormitoire.wordpress.com 
"A standpoint is an intellectual horizon of radius zero".  (A. Einstein) 

On Tuesday, April 17, 2018, 3:37:01 a.m. EDT, Sven Van Caekenberghe 
 wrote:  
 
 Hi,

The dictionary 

 OSPlatform current environment

contains a copy of the OS's environment variables (more correctly of the VM 
process), as key/value pairs. 

These are obtained via the following system calls:

on macOS & *nix

  LIBC environ

on Windows

  KERNEL32 GetEnvironmentStrings

It is however a bit unclear how these are encoded. On macOS & *nix that seems 
to be UTF8, on Windows there are some reports that it appears to be Latin1 - 
but both might be locale specific, I don't know either way.

Does anyone know for sure ?

I furthermore think that OSEnvironment and its subclasses, who do this call, 
should be responsible for decoding the C strings into proper Pharo strings, and 
not leave that responsibility to its users.

Fundamentally, in the following, the decoding is still not done correctly and 
that is wrong/confusing IMHO.

$ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations' 
{'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 
'SHELL'->'/bin/bash'. 
'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl8gn/T/'. 
'FOO'->'benoît'. 
'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
 'TERM_PROGRAM_VERSION'->'404'. 
'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 
'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 
'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 
'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 
'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'. 
'_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. 
'__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}

Of course, if we change this, we will need to fix callers.

Opinions ?

Sven

PS: Furthermore, I note that there is a subtle difference in how $FOO and $PWD 
in the above are UTF-8 encoded. In the former, normalisation was done, in the 
latter not. Maybe that could lead to problems (when comparing/composing them). 
This is a difficult/complex subject 
(https://medium.com/concerning-pharo/an-implementation-of-unicode-normalization-7c6719068f43).


  --- End Message ---

Re: [Pharo-dev] [Pharo-Launcher] call for tests on Windows

[Christophe Demarey]

Hi Phil, Eliot,

> Le 16 avr. 2018 à 23:58, Eliot Miranda  a écrit :
> 
> Hi Phil,
> 
> On Mon, Apr 16, 2018 at 12:54 PM, p...@highoctane.be 
>  mailto:p...@highoctane.be>> 
> wrote:
> I have downloaded the 
> https://github.com/pharo-project/pharo-vm/blob/master/opensmalltalk-vm/build.win32x86/pharo.cog.spur.lowcode/Pharo.exe.manifest
>  
> 
>  and put it in the folder that contains the PharoLauncher Pharo.exe and then 
> the pointers are correct again (they have their masking done correctly.
> 
> Good!  
>  
> So, there is a packaging issue somewhere for Pharo Windows VMs as downloaded 
> by the Laucher.
> 
> Indeed, we must include the manifest files on Windows.

I did not get this problem. Is this manifest file new? Is there only one 
manifest file to include or many?

Thanks for reporting this issue.
Christophe

[Pharo-dev] Environment variables encoding ?

[Sven Van Caekenberghe]

Hi,

The dictionary 

 OSPlatform current environment

contains a copy of the OS's environment variables (more correctly of the VM 
process), as key/value pairs. 

These are obtained via the following system calls:

on macOS & *nix

  LIBC environ

on Windows

  KERNEL32 GetEnvironmentStrings

It is however a bit unclear how these are encoded. On macOS & *nix that seems 
to be UTF8, on Windows there are some reports that it appears to be Latin1 - 
but both might be locale specific, I don't know either way.

Does anyone know for sure ?

I furthermore think that OSEnvironment and its subclasses, who do this call, 
should be responsible for decoding the C strings into proper Pharo strings, and 
not leave that responsibility to its users.

Fundamentally, in the following, the decoding is still not done correctly and 
that is wrong/confusing IMHO.

$ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations' 
{'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 
'SHELL'->'/bin/bash'. 
'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl8gn/T/'. 
'FOO'->'benoît'. 
'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
 'TERM_PROGRAM_VERSION'->'404'. 
'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 
'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 
'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 
'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 
'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'. 
'_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. 
'__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}

Of course, if we change this, we will need to fix callers.

Opinions ?

Sven

PS: Furthermore, I note that there is a subtle difference in how $FOO and $PWD 
in the above are UTF-8 encoded. In the former, normalisation was done, in the 
latter not. Maybe that could lead to problems (when comparing/composing them). 
This is a difficult/complex subject 
(https://medium.com/concerning-pharo/an-implementation-of-unicode-normalization-7c6719068f43).

[Pharo-dev] [Pharo 7.0-dev] Build #772: 21715-When-asking-for-the-context-in-a-after-node-on-a-method-return-context-of-method-not-the-ensure-block

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #772 was: SUCCESS.

The Pull Request #1205 was integrated: 
"21715-When-asking-for-the-context-in-a-after-node-on-a-method-return-context-of-method-not-the-ensure-block"
Pull request url: https://github.com/pharo-project/pharo/pull/1205

Issue Url: https://pharo.fogbugz.com/f/cases/21715
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/772/