Re: [Pharo-dev] Moving from mc to tonel?

[Peter Uhnák]

> I suspect it's related to the large number of commits in my repo. I made
> some tweaks and succeeded to create the fast-import file. But I get:
>

All files in a single commit will be held temporarily in MemoryStore, so
unless you have GBs of code in _single_ commit it shouldn't be a problem.
There's also a possiblity that Pharo was going too fast and didn't manage
to GC it in time or there's a memory leak.


>
> fatal: Unsupported command: .
> fast-import: dumping crash report to .git/fast_import_crash_10301
>
> Do you recognize this error?
>


Please look into the mentioned file, it should give you a context for the
error (and then you can look into the import file around the same context).

Peter

Re: [Pharo-dev] Moving from mc to tonel?

[Sven Van Caekenberghe]

> On 5 Dec 2017, at 08:34, Alistair Grant  wrote:
> 
> On 5 December 2017 at 03:41, Martin Dias  wrote:
>> I suspect it's related to the large number of commits in my repo. I made
>> some tweaks and succeeded to create the fast-import file. But I get:
>> 
>> fatal: Unsupported command: .
>> fast-import: dumping crash report to .git/fast_import_crash_10301
>> 
>> Do you recognize this error?
>> I will check my changes tweaking the git-migration tool to see if I modified
>> some behavior my mistake...
> 
> I had the same error just last night.
> 
> In my case, it turned out to be a non-UTF8 encoded character in one of
> the commit messages.
> 
> I tracked it down by looking at the crash report and searching for a
> nearby command.  I've deleted the crash reports now, but I think it
> was the number associated with a mark command that got me near the
> problem character in the fast-import file.
> 
> I also modified the code to halt whenever it found a non-UTF8
> character.  I'm sure there are better ways to do this, but:
> 
> 
> GitMigrationCommitInfo>>inlineDataFor: aString
> 
>| theString |
>theString := aString.
>"Ensure the message has valid UTF-8 encoding (this will raise an
> error if it doesn't)"
>[ (ZnCharacterEncoder newForEncoding: 'utf8') decodeBytes: aString
> asByteArray ]
>on: Error
>do: [ :ex | self halt: 'Illegal string encoding'.
>theString := aString select: [ :each | each asciiValue
> between: 32 and: 128 ] ].
>^ 'data ' , theString size asString , String cr , (theString
> ifEmpty: [ '' ] ifNotEmpty: [ theString , String cr ])

There is also ByteArray>>#utf8Decoded (as well as String>>#utf8Encoded), both 
using the newer ZnCharacterEncoders. So you could write it shorter as:

  aString asByteArray utf8Decoded

Instead of Error you could use the more intention revealing 
ZnCharacterEncodingError.

Apart from that, and I known you did not create this mess, encoding a String 
into a String, although it can be done, is so wrong. You encode a String (a 
collection of Characters, of Unicode code points) into a ByteArray and you 
decode a ByteArray into a String.

Sending #asByteArray to a String in almost always wrong, as is sending 
#asString to a ByteArray. These are implicit conversions (null conversions, 
like ZnNullEncoder) that only work for pure ASCII or Latin1 (iso-8859-1), but 
not for the much richer set of Characters that Pharo supports. So these will 
eventually fail, one day, in a country far away.

> Cheers,
> Alistair
> 

Re: [Pharo-dev] Moving from mc to tonel?

[Alistair Grant]

On 5 December 2017 at 03:41, Martin Dias  wrote:
> I suspect it's related to the large number of commits in my repo. I made
> some tweaks and succeeded to create the fast-import file. But I get:
>
> fatal: Unsupported command: .
> fast-import: dumping crash report to .git/fast_import_crash_10301
>
> Do you recognize this error?
> I will check my changes tweaking the git-migration tool to see if I modified
> some behavior my mistake...

I had the same error just last night.

In my case, it turned out to be a non-UTF8 encoded character in one of
the commit messages.

I tracked it down by looking at the crash report and searching for a
nearby command.  I've deleted the crash reports now, but I think it
was the number associated with a mark command that got me near the
problem character in the fast-import file.

I also modified the code to halt whenever it found a non-UTF8
character.  I'm sure there are better ways to do this, but:


GitMigrationCommitInfo>>inlineDataFor: aString

| theString |
theString := aString.
"Ensure the message has valid UTF-8 encoding (this will raise an
error if it doesn't)"
[ (ZnCharacterEncoder newForEncoding: 'utf8') decodeBytes: aString
asByteArray ]
on: Error
do: [ :ex | self halt: 'Illegal string encoding'.
theString := aString select: [ :each | each asciiValue
between: 32 and: 128 ] ].
^ 'data ' , theString size asString , String cr , (theString
ifEmpty: [ '' ] ifNotEmpty: [ theString , String cr ])



Cheers,
Alistair

Re: [Pharo-dev] Pharo newsletter archive

[Tudor Girba]

+1

Doru


> On Dec 5, 2017, at 7:54 AM, Stephane Ducasse  wrote:
> 
> Yes good idea!
> Tx
> 
> On Mon, Dec 4, 2017 at 7:52 PM, Torsten Bergmann  wrote:
>> Hi,
>> 
>> the newsletter archive is now available on
>> 
>>  "newsletter.pharo.org"
>> 
>> which is more easy to remember.
>> 
>> Thanks Marcus for setting up the redirect!
>> 
>> Bye
>> T.
>> 
> 

--
www.tudorgirba.com
www.feenk.com

"When people care, great things can happen."

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo TechTalk 21 Nov: Discord Demo

[Tudor Girba]

Thanks!

Doru


> On Nov 23, 2017, at 8:26 PM, Juraj Kubelka  wrote:
> 
> Hi, the new version is available here: 
> https://www.youtube.com/watch?v=y1EzOnfHUe0 
> 
> Hopefully it helps :-)
> Juraj
> 
>> On Nov 22, 2017, at 15:45, Sean P. DeNigris  wrote:
>> 
>> Juraj Kubelka wrote
>>> I will upload recording I have on my computer. It is clear.
>> 
>> Thanks!
>> 
>> 
>> 
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>> 
> 

--
www.tudorgirba.com
www.feenk.com

"Value is always contextual."

Re: [Pharo-dev] Pharo newsletter archive

[Stephane Ducasse]

Yes good idea!
Tx

On Mon, Dec 4, 2017 at 7:52 PM, Torsten Bergmann  wrote:
> Hi,
>
> the newsletter archive is now available on
>
>   "newsletter.pharo.org"
>
> which is more easy to remember.
>
> Thanks Marcus for setting up the redirect!
>
> Bye
> T.
>

Re: [Pharo-dev] Moving from mc to tonel?

[Martin Dias]

I suspect it's related to the large number of commits in my repo. I made
some tweaks and succeeded to create the fast-import file. But I get:

fatal: Unsupported command: .
fast-import: dumping crash report to .git/fast_import_crash_10301

Do you recognize this error?
I will check my changes tweaking the git-migration tool to see if I
modified some behavior my mistake...

cheers,
Martín

On Fri, Dec 1, 2017 at 1:42 PM, Martin Dias  wrote:

> Julien, I dodn't know it but anyway my issue was just in the step
> smalltalkhub -> filtree using the migration tool.
>
> Peter, yes it's public. This is the code I used:
>
> 
>
> migration := GitMigration on: 'MartinDias/Epicea'.
> migration cacheAllVersions.
> migration authors: {
> 'md' -> #('Martin Dias' '').
> 'MD' -> #('Martin Dias' '').
> 'MartinDias' -> #('Martin Dias' '').
> 'PabloTesone' -> #('Pablo Tesone' '').
> 'PavelKrivanek' -> #('Pavel Krivanek' '').
> 'DamienCassou' -> #('Damien Cassou' '').
> 'DenisKudryashov' -> #('Denis Kudryashov' '').
> 'GustavoSantos' -> #('Gustavo Santos' '').
> 'ThomasHeniart' -> #('Thomas Heniart' '').
> 'LucasGodoy' -> #('Lucas Godoy' '').
> 'StephaneDucasse' -> #('Stephane Ducasse' '').
> 'VincentBlondeau' -> #('Vincent Blondeau' '').
> 'AlbertoBacchelli' -> #('Alberto Bacchelli' '').
> 'SkipLentz' -> #('Skip Lentz' '').
> 'TheIntegrator' -> #('The Integrator' '').
> }.l
> migration
> fastImportCodeToDirectory: 'src'
> initialCommit: '18a30d8d'
> to: './epicea-fast-import.txt'
>
> --
>
> Martín
>
> On Thu, Nov 30, 2017 at 2:14 PM, Julien 
> wrote:
>
>> If you use Traits and want to have Epicea loadable from GitHub on Pharo
>> 6.1, it will not work.
>>
>> There is an issue when you try to load a Trait in Tonel format in the
>> image.
>>
>> ---
>> Julien Delplanque
>> Doctorant à l’Université de Lille 1
>> http://juliendelplanque.be/phd.html
>> Equipe Rmod, Inria
>> Bâtiment B 40, avenue Halley 59650
>> 
>>  Villeneuve
>> 
>>  d'Ascq
>> 
>> Numéro de téléphone: +333 59 35 86 40
>>
>> Le 6 nov. 2017 à 01:42, Martin Dias  a écrit :
>>
>> Hi all, I'm considering to move Epicea (+Ombu+Hiedra) to github. I read
>> that tonel is the new format and there is Peter's git-migration, which
>> could be the tool to do it.
>>
>> Is it good idea to move now?
>> Is it the right tool?
>>
>> Bests,
>> Martín
>>
>>
>>
>

Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap)

[Ben Coman]

Hi Eliot,

On 4 December 2017 at 23:44, Eliot Miranda  wrote:
>
> Alas a forwarding object has two fields that need to be set in separate
words.
> The class index in the header must be the forwarder class index,
> and the first word of the object body must point to the forwarder.

Good to better understand that.
Just restating after condensing most of the thread, only the Main-thread
would convert objects to forwarders.


> Reverting obviously requires two writes too. So atomic (un)forwarding is
not possible.

sorry, just double-checking since I realised I erred in my example below,
which may have mislead your response.

IIUC(now)... while "creating" a forwarder involves writing separately to
object-header and object-body.
wouldn't "reverting" only involve updating an object-pointer in a slot to
the real-object's new location?
And that would be a single word operation on both 32bit and 64bit platforms?


> >> - Currently the marking phase removes all forwarders and I would like
incremental marking to maintain the same invariant (forwarders are always
white).
> >
> > A concurrent-marking thread could essentially do the same.
> > i. From shared memory load forward header Fh from object-slot
> >Fheader <== object-slot
> > ii. Follow forwarder to real-object, store into local temporary
> >Rheader <== flattened/followed pointer
> >
> > iii. Atomic compare-and-swap R back into object-slot,
> >object-slot <== if Fheader then Rheader

whoops, corrected...

i. From shared memory, concurrent marking-scan has loaded an
object-pointer from object-slot.  If it points to a forwarding object ...
  Opointer <== object-slot (as part of normal marking scan)
  if *Opointer isForwardingObject {
Fpointer <== Opointer (redundant, just for clarity)

ii. Follow that to real-object, store into local temporary
  Rpointer <== followed Fpointer forwarding chain to real
object.

iii. Atomic compare-and-swap Rpointer back into object-slot
  object-slot <== if still Fpointer then Rpointer



When (iii.) fails
* If I'm the Main-thread, I'm in the middle of handling a failed
forwarder-check
  and can infer the GC-thread already did what I was going to do.
  I can be certain that..."object-slot == Rpointer" (since thats the
only object-slot mutation GC-threads can do)
  so its fine to continue to my usual post-flattening retry

* If I'm a GC-thread, either:
 * the Main-thread flattened the forwarder, or
 * the Main-thread changed which object the slot holds
   In either case, just re-read the slot

A race scenario to consider is where the Main-thread converts an object
to a forwarder simultaneous with a GC-thread flattening one of its slots.
The object copy in its new location may miss that update.
But in terms of the object graph that seems not a problem, since the copy
without that update is still consistent.
Infrequently just a little bit of work is lost.

I'm contemplating other race scenarios around marking, but won't distract
with them for now.

cheers -ben

Re: [Pharo-dev] Bloc Feedback

[Ben Coman]

On 5 December 2017 at 05:19, Stephane Ducasse 
wrote:

> Hi Aliaksei
>
> Read again this sentence :)
>
>  "BlPostAction" - a valuable that is guaranteed to be evaluated in the
> beginning of every frame before anything else.
>
> MAy be BlPostAction should be renamed BlEarlyAction
>

BlPreAction


>
>
>
> On Sat, Dec 2, 2017 at 5:26 PM, Aliaksei Syrel 
> wrote:
> > Hi,
> >
> > An object that is called now "Animation" in Bloc is not necessarily
> bound to
> > the visual element.
> > The correct name for the root class of "BlBaseAnimation" should be
> > "BlPostAction" - a valuable that is guaranteed to be evaluated in the
> > beginning of every frame before anything else. We have already a deferred
> > action but they are only evaluated when there is enough time between
> frames.
> > "BlBaseAnimation" should be renamed to something line
> "BlElementAnimation"
> > that is guaranteed to be bound to a visual element and stops itself if
> > element is no more in the scene graph by checking that explicitly before
> > doing every animation step.
> >
> > Cheers,
> > Alex
> >
> > On 2 December 2017 at 16:35, Sean P. DeNigris 
> wrote:
> >>
> >> Aliaksei Syrel wrote
> >> > ...
> >> >  Also, what is the lifetime of an infinite animation?
> >> > ...
> >> > Do you have any particular use-case for animations?
> >>
> >> I hacked together a simplified stepping animation (may not be in the
> right
> >> place in the class hierarchy). Here's a pull request to discuss
> >> https://github.com/pharo-graphics/Bloc/pull/37
> >>
> >>
> >>
> >> -
> >> Cheers,
> >> Sean
> >> --
> >> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.
> html
> >>
> >
>
>

Re: [Pharo-dev] Polymorphism between Strings and FileReference

[Stephane Ducasse]

To me stringOfFileReference is a bad smell.



On Fri, Dec 1, 2017 at 12:04 PM, Cyril Ferlicot
 wrote:
> On Fri, Dec 1, 2017 at 11:55 AM, Alistair Grant  wrote:
>> Hi Cyril,
>>
>> The normal way to handle this is #asFileReference, so you can just have:
>>
>> aStringOrReference asFileReference readStreamDo: ...
>>
>
> In that case, if it is a String like 'test', we will get a readStream
> on the file './test' and not on the string 'test'. This is not what I
> want.
>
> What I want is this for example:
>
>
>
> MyClass>>addComaAtTheEndOf: aStringOrFileReference
>   "If the argument is a String, add a coma at the end. If the
> parameter is a file, add a coma at the end of its contents"
>
>   aStringOrFileReference writeStreamDo: [ :s | s atEnd; nextPut: $, ]
>
>
>
> If I use your code, it will not add a coma at the end of the string
> parameter, but at the end of the file represented by the string if it
> exists.
>
>> Cheers,
>> Alistair
>>
>
>
>
> --
> Cyril Ferlicot
> https://ferlicot.fr
>
> http://www.synectique.eu
> 2 rue Jacques Prévert 01,
> 59650 Villeneuve d'ascq France
>

Re: [Pharo-dev] [TechTalk] Topics Wanted for 2018!

[Stephane Ducasse]

I could show how to setup a pillar project.

Stef

On Fri, Dec 1, 2017 at 11:58 AM, Marcus Denker  wrote:
> Hi,
>
> As you know, we are trying to have one Pharo tech-talk per month.
>
> The idea is to use this for
> -> mini “Lectures”
> -> Demos of interesting Projects
> -> Just some interesting topic to discuss with audio chat on Discord
>
> You can see the past topics here:
>
> http://pharo.org/TechTalk
>
> We will soon define the dates for the first half of 2018. If you want to do
> a techtalk
> yourself (give a presentation / Demo or moderate a discussion), feel free to
> contact
> me! You are able to pick a date that works best for you.
>
>
> Marcus

Re: [Pharo-dev] Bloc and Gestures

[Stephane Ducasse]

I really hope that we will be able to take advantage of the work done
by merwan and jb on gestures
but yes we need sdl20 working.

Stef

On Sat, Dec 2, 2017 at 1:02 PM, Aliaksei Syrel  wrote:
> Hi Sean,
>
> The main problem of gestures is that they can only theoretically work with
> OS-Window (SDL2 or other windowing library).
> With morphic I can not even find how to add support of horizontal scrolling
> using touchpad...
>
> For this reason those gestures are hierarchy placeholder and are not used
> anywhere.
> As for ETA ... 3 months maybe :) I think we will work on gestures as soon as
> bloc will officially supports SDL2 windows.
>
> Cheers,
> Alex

Re: [Pharo-dev] Bloc Feedback

[Stephane Ducasse]

Hi Aliaksei

Read again this sentence :)

 "BlPostAction" - a valuable that is guaranteed to be evaluated in the
beginning of every frame before anything else.

MAy be BlPostAction should be renamed BlEarlyAction



On Sat, Dec 2, 2017 at 5:26 PM, Aliaksei Syrel  wrote:
> Hi,
>
> An object that is called now "Animation" in Bloc is not necessarily bound to
> the visual element.
> The correct name for the root class of "BlBaseAnimation" should be
> "BlPostAction" - a valuable that is guaranteed to be evaluated in the
> beginning of every frame before anything else. We have already a deferred
> action but they are only evaluated when there is enough time between frames.
> "BlBaseAnimation" should be renamed to something line "BlElementAnimation"
> that is guaranteed to be bound to a visual element and stops itself if
> element is no more in the scene graph by checking that explicitly before
> doing every animation step.
>
> Cheers,
> Alex
>
> On 2 December 2017 at 16:35, Sean P. DeNigris  wrote:
>>
>> Aliaksei Syrel wrote
>> > ...
>> >  Also, what is the lifetime of an infinite animation?
>> > ...
>> > Do you have any particular use-case for animations?
>>
>> I hacked together a simplified stepping animation (may not be in the right
>> place in the class hierarchy). Here's a pull request to discuss
>> https://github.com/pharo-graphics/Bloc/pull/37
>>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
>>
>

[Pharo-dev] [Pharo 7.0-dev] Build #358: 20786-halt-messages-are-not-highlighted-in-red-as-the-other-flavours-of-halt-messages

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #358 was: SUCCESS.

The Pull Request #569 was integrated: 
"20786-halt-messages-are-not-highlighted-in-red-as-the-other-flavours-of-halt-messages"
Pull request url: https://github.com/pharo-project/pharo/pull/569

Issue Url: https://pharo.fogbugz.com/f/cases/20786
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/358/

[Pharo-dev] Pharo newsletter archive

[Torsten Bergmann]

Hi,

the newsletter archive is now available on

  "newsletter.pharo.org"

which is more easy to remember.

Thanks Marcus for setting up the redirect!

Bye
T.

Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap)

[Eliot Miranda]

Hi Ben,

> On Dec 4, 2017, at 6:38 AM, Ben Coman  wrote:
> 
>> On 4 December 2017 at 15:47, Clément Bera  wrote:
>> 
>> Here are some ideas/things to say on the tip of my head:
>> - Changing an object to a forwarding object is non atomic (we need to 
>> maintain at least stack invariant)
> 
> Thats because the whole multiword object has to be copied.
> What about the reverse, flattening a forwarding-object back into the
> real-object?
> I presume only the one word of the object-pointer is updated.
> btw, are we safe if competing threads write the *same* data to a slot,

Alas a forwarding object has two fields that need to be set in separate words.  
The class index in the header must be the forwarder class index, and the first 
word of the object body must point to the forwarder.  Reverting obviously 
requires two writes too.  So atomic (un)forwarding is not possible.


> 
> 
> Anyway my idea was the GC-thread would only *identify* objects to be
> moved, and queue them
> for the Main-thread to chip away at, so only one thread is converting
> objects to forwarding objects.
> 
> 
>> - To decrease the pauses in full GC I have 2 plans:
>> -- incremental marking (split the mark pause in multiple small pauses): Not 
>> implemented right now.
>> -- selective compaction (compacts only part of the heap instead of all the 
>> heap and sweeps the rest, similar to G1, but uses forwarders instead of lots 
>> of card marking metadata): I implemented SpurSweeper which only sweeps but 
>> works very well.
>> - Currently the marking phase removes all forwarders and I would like 
>> incremental marking to maintain the same invariant (forwarders are always 
>> white).
> 
> A concurrent-marking thread could essentially do the same.
> i. From shared memory load forwarder F from object-slot
>F <== object-slot
> 
> ii. Follow forwarder to real-object, store into temporary R
>R <== flattened/followed pointer
> 
> iii. Atomic compare-and-swap R back into object-slot,
>object-slot <== if F then R
> 
> When (iii.) fails
> * If I'm the Main-thread, then an other-thread already did what I wanted,
>   and since thats the *only* mutation other-threads can do to an object-slot.
>   I am certain..."object-slot == R",   so since I'm handling a
> failed forwarding-check, just continue with the normal retry.
> * If I'm an other-thread, no hurry.  Read the object-slot again and it
> "should" be a real-object, otherwise just keep trying until I get one.
> 
> So the question is... When using forwarders for compaction,
> how often would fail forwarder-check fail...
>  * if there was one thread on its own; versus
>  * in the Main-thread if an other-thread had already flattened many of them
> 
> 
>> - In general, Concurrent marking and sweeping have been implemented 
>> everywhere, but no concurrent compaction. For compaction you can make it 
>> selective (compact only part of the heap and the part which needs it the 
>> most) like I suggest and like in G1, which decreases considerably compaction 
>> pause time. Work on concurrent compaction is state of the art and not in 
>> production everywhere, see for example
> 
> IIUC not many other languages use forwarding pointer like we do, and
> these seem like a real advantage
> to compact incrementally and concurrently.
> 
> 
>> No more time right now.
> 
> I recognise it wasn't a great format.
> I really appreciate the time you could spare.
> 
> cheers -ben
> 

[Pharo-dev] feenk log

[Tudor Girba]

Hi,

Here is an update of the work on Bloc and GT:

Bloc:
- Bezier curves (cubic and quadratic).
https://twitter.com/feenkcom/status/93476564021504
https://twitter.com/feenkcom/status/934856909001252865

- Dragging is now possible through a BlDragHandler that can be attached to any 
element.

- Pixel accurate line heads and tails
https://twitter.com/feenkcom/status/936704646290305025

- More flexible scalable element.
https://twitter.com/feenkcom/status/934003681539907585

- Improved scalability of the rendering. For example, on Mac, we are on par 
with the speed of Firefox.
https://twitter.com/feenkcom/status/931462535861108736

- Sparta is now able to load custom fonts.
https://twitter.com/feenkcom/status/930919637214285824

- Text insertion works now much better. Deletion and Selection still require 
more work.

- The rope data structure can now be better inspected.
https://twitter.com/feenkcom/status/937298739584172032

- Sparta/Moz2D has a visual loader that also provides more specific details in 
case of errors.
https://twitter.com/feenkcom/status/936501909581688832

- Moz2D is now installed by default on Linux 64b.


GT:

- The GT Visualizer sub-project now contains two tools:

-- GT Mondrian is a graph visualization engine that is both performant and 
shows how Bloc works well for visualizations. More details in a follow up post.
https://twitter.com/feenkcom/status/937682660704563202

-- GT Connector is a new tool for browsing connected examples. More details 
about it in a follow up post.
https://twitter.com/feenkcom/status/936109463462965248


Enjoy,
The feenk team

--
www.tudorgirba.com
www.feenk.com

"If you can't say why something is relevant, 
it probably isn't."

Re: [Pharo-dev] Alternative bytecode sets demo request

[Eliot Miranda]

Hi Ben,

> On Dec 4, 2017, at 4:17 AM, Ben Coman  wrote:
> 
>> On 4 December 2017 at 10:48, Eliot Miranda  wrote:
>> Hi Ben,
>> 
>>> On Sun, Dec 3, 2017 at 6:08 PM, Ben Coman  wrote:
>>> 
>>> I'm shooting in the dark here since its a bit hard to grasp...
>>> a. The implications of support alternative bytecode sets
>>>* Do the alternative bytecode sets run in parallel to normal Pharo
>>> bytecodes?
>> 
>> 
>> Yes.  There's a single bit in the header of a compiled method that selects
>> between two bytecode sets.  In current Squeak and Pharoi VMs, if the bit is
>> unset then the normal bytecode set is used, and if the nit is set, the Sista
>> bytecode set is used.
> 
> That clarifies things a lot.  So I guess the main advantage of this bit
> is making it easier to:
> a. incrementally develop and test new bytecodes?
> b. transition the Image between bytecode sets?

Exactly.

>>>* Can the bytecode sets be loaded adhoc/dynamically into an Image?
>>>   or only compiled into the VM?
>> 
>> 
>> Only compiled into the VM.
>> 
>>> 
>>> b. How to implement/work with alternative bytecodes
>> 
>> 
>> In Squeak there are class-side accessors in CompiledCode.  See CompiledCode
>> class>>installSecondaryBytecodeSet:.  Note that the support isn't quite
>> finished yet.  I hope to have the Sista vytecde set ready for the next
>> release of Squeak.  Clément is providing it in Pharo.
>> 
>> In Squeak the BytecodeEncoder hierarchy implements the necessary support for
>> multiple bytecode sets.  Via CompiledCode
>> class>>installSecondaryBytecodeSet: the system is informed as to which
>> bytecode set to use.  This must match the bytecode set(s) that are in the
>> VM.  Classes beneath BytecodeEncoder, in particular EncoderForV3PlusClosures
>> and EncoderForSistaV1 (in package BytecodeSets at
>> http://source.squeak.org/VMMaker) encode the current and the Sista sets
>> respectively.
> 
> Can more than two bytecode sets be compiled into the one VM?
> i.e. Is a bytecode-set hard coded into position,
> or is it more like a variable slot that can select between several
> bytecode sets compiled into the VM.  For example, if someone wanted
> to experiment with implementing Ethereum bytecodes, would they first need
> to rip out either EncoderForV3PlusClosures or EncoderForSistaV1?

It's hard coded.  There could be more sets.  This is Claus Gittinger's idea and 
in his VM there are two bits, allowing 4 sets.  The implementation is simple.  
A variable holds the bytecode index times 256, and this is added to the 
bytecode on each dispatch in either the interpreter or the JIT.  So in Claus' 
VM the bytecodes range from 0 to 1023.  The variable must be set on every send 
and return.  Obviously in jitted code it is effectively free.

We were short of header bits and only needed one extra set so we support only 
two.

> 
> 
> Now I made the mistake once of thinking it was Sista introducing this
> bytecode-bit,
> (btw, how do you normally refer to it?) but IIUC Sista is just making use
> of the feature.  So I'm curious historically when the bytecode-bit was
> introduced?

I added it with Claus when we were working on a project together in 2012.  It 
was first used for Newspeak in 2013 or 2014 when I defined a set that supported 
the full range of Newspeak sends and lifted the number of literals and branch 
displacement limits.

> 
> cheers -ben
> 
>> 
>>> To help (b.), would it be feasible to blog a mini-demo showing how to
>>> install/use alternative bytecodes? maybe for something like this simple RPN
>>> calculator substitute for "arith.c"...
>>> * https://github.com/philipaconrad/mini-vm/tree/master/examples/arith
>>> *
>>> https://github.com/philipaconrad/mini-vm/blob/master/examples/arith/arith.c
>> 
>> 
>> That's not how this will work.  Essentially it is transparent.  Once the
>> bytecode set support is installed in the compiler one simply compiles
>> Smalltalk (or anything else that produces parse trees that can be output to
>> compiled methods via the BytecodeEncoder hierarchy) and the method will be
>> output in the currently selected bytecode set.  So it's not something one
>> plays around with, unless that is, one is developing the bytecode set in the
>> Vm simulator.
> 

[Pharo-dev] [Pharo 7.0-dev] Build #357: 20781-update-to-iceberg-064

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #357 was: SUCCESS.

The Pull Request #560 was integrated: "20781-update-to-iceberg-064"
Pull request url: https://github.com/pharo-project/pharo/pull/560

Issue Url: https://pharo.fogbugz.com/f/cases/20781
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/357/

Re: [Pharo-dev] Alternative bytecode sets demo request

[Ben Coman]

On 4 December 2017 at 10:48, Eliot Miranda  wrote:
> Hi Ben,
>
> On Sun, Dec 3, 2017 at 6:08 PM, Ben Coman  wrote:
>>
>> I'm shooting in the dark here since its a bit hard to grasp...
>> a. The implications of support alternative bytecode sets
>> * Do the alternative bytecode sets run in parallel to normal Pharo
>> bytecodes?
>
>
> Yes.  There's a single bit in the header of a compiled method that selects
> between two bytecode sets.  In current Squeak and Pharoi VMs, if the bit is
> unset then the normal bytecode set is used, and if the nit is set, the Sista
> bytecode set is used.

That clarifies things a lot.  So I guess the main advantage of this bit
is making it easier to:
a. incrementally develop and test new bytecodes?
b. transition the Image between bytecode sets?






>> * Can the bytecode sets be loaded adhoc/dynamically into an Image?
>>or only compiled into the VM?
>
>
> Only compiled into the VM.
>
>>
>> b. How to implement/work with alternative bytecodes
>
>
> In Squeak there are class-side accessors in CompiledCode.  See CompiledCode
> class>>installSecondaryBytecodeSet:.  Note that the support isn't quite
> finished yet.  I hope to have the Sista vytecde set ready for the next
> release of Squeak.  Clément is providing it in Pharo.
>
> In Squeak the BytecodeEncoder hierarchy implements the necessary support for
> multiple bytecode sets.  Via CompiledCode
> class>>installSecondaryBytecodeSet: the system is informed as to which
> bytecode set to use.  This must match the bytecode set(s) that are in the
> VM.  Classes beneath BytecodeEncoder, in particular EncoderForV3PlusClosures
> and EncoderForSistaV1 (in package BytecodeSets at
> http://source.squeak.org/VMMaker) encode the current and the Sista sets
> respectively.

Can more than two bytecode sets be compiled into the one VM?
i.e. Is a bytecode-set hard coded into position,
or is it more like a variable slot that can select between several
bytecode sets compiled into the VM.  For example, if someone wanted
to experiment with implementing Ethereum bytecodes, would they first need
to rip out either EncoderForV3PlusClosures or EncoderForSistaV1?


Now I made the mistake once of thinking it was Sista introducing this
bytecode-bit,
(btw, how do you normally refer to it?) but IIUC Sista is just making use
of the feature.  So I'm curious historically when the bytecode-bit was
introduced?

cheers -ben

>
>> To help (b.), would it be feasible to blog a mini-demo showing how to
>> install/use alternative bytecodes? maybe for something like this simple RPN
>> calculator substitute for "arith.c"...
>> * https://github.com/philipaconrad/mini-vm/tree/master/examples/arith
>> *
>> https://github.com/philipaconrad/mini-vm/blob/master/examples/arith/arith.c
>
>
> That's not how this will work.  Essentially it is transparent.  Once the
> bytecode set support is installed in the compiler one simply compiles
> Smalltalk (or anything else that produces parse trees that can be output to
> compiled methods via the BytecodeEncoder hierarchy) and the method will be
> output in the currently selected bytecode set.  So it's not something one
> plays around with, unless that is, one is developing the bytecode set in the
> Vm simulator.

[Pharo-dev] [Pharo 7.0-dev] Build #356: 20793-During-Bootstrap-if-a-baseline-installation-fails-it-should-cancel-the-process

[ci-pharo-ci-jenkins2]

There is a new Pharo build available!

The status of the build #356 was: FAILURE.

The Pull Request #567 was integrated: 
"20793-During-Bootstrap-if-a-baseline-installation-fails-it-should-cancel-the-process"
Pull request url: https://github.com/pharo-project/pharo/pull/567

Issue Url: https://pharo.fogbugz.com/f/cases/20793
Build Url: 
https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%20and%20branch%20Pipeline/job/development/356/

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

[Mariano Martinez Peck]

Hi Luke,

Not only for the details of a particular proxy implementation like Ghost,
but also because of all the discussions around it,  I think this journal
paper [1] we wrote may be of help to you.

Cheers,

[1] https://hal.inria.fr/hal-01081236


On Mon, Dec 4, 2017 at 5:51 AM, Luke Gorrie  wrote:

> Thanks Ben & Henrik for the ideas!
>
> I will scratch my head a bit and then try to do something sensible. :-)
>
>
>


-- 
Mariano
http://marianopeck.wordpress.com

Re: [Pharo-dev] Pharo Launcher endless loop opening debuggers in Pharo 6.1

[Christophe Demarey]

Hi,

I cannot reproduce this bug.
Do you have the log or the stack trace?

> Le 2 déc. 2017 à 14:51, Bernhard Pieber  a écrit :
> 
> I ran into an annoying problem using the latest Pharo 6.1 (stable) from Pharo 
> Launcher. I am on macOS Sierra Version 10.12.6.
> 
> Steps to reproduce:
> 1. Open Pharo Launcher
> 2. Create new Pharo 6.1 (stable) image from the official distributions, named 
> Test
> 3. Launch the Test image
> 4. Open a File Browser
> 5. Right click on the selected Test directory in the left file tree pane.
> 6. Debuggers start opening on KeyNotFound: key #arg4 not found in Dictionary
> 7. Image hangs
> 
> Interestingly the problem does not happen if I create the image using „curl 
> get.pharo.org | bash“ and start it with „./pharo-ui Pharo.image“ although 
> About Pharo shows the same information:
> Pharo 6.0
> Latest update: #60523
> 
> Can someone please confirm the bug?
> 
> Bernhard

Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap)

[Benoit St-Jean via Pharo-dev]

--- Begin Message ---
Perhaps the solution resides in settings you can tweak from within Smalltalk?  
Something like MemoryPolicy you have in VisualWorks for memory & garbage 
collection management. I don't think fixing the behavior of memory management & 
garbage collection (marking, sweeping, etc) in the VM is the way to go.  
There's an awful lot of different needs for different applications out there !
 - 
Benoît St-Jean 
Yahoo! Messenger: bstjean 
Twitter: @BenLeChialeux 
Pinterest: benoitstjean 
Instagram: Chef_Benito
IRC: lamneth 
Blogue: endormitoire.wordpress.com 
"A standpoint is an intellectual horizon of radius zero".  (A. Einstein)

  From: Clément Bera 
 To: Squeak Virtual Machine Development Discussion 
 
Cc: Pharo Development List 
 Sent: Monday, December 4, 2017 2:49 AM
 Subject: Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the 
roadmap)
   
hum...
The mail is very long so I did not read all of it.
Here are some ideas/things to say on the tip of my head:- Changing an object to 
a forwarding object is non atomic (we need to maintain at least stack 
invariant)- To decrease the pauses in full GC I have 2 plans:-- incremental 
marking (split the mark pause in multiple small pauses): Not implemented right 
now.-- selective compaction (compacts only part of the heap instead of all the 
heap and sweeps the rest, similar to G1, but uses forwarders instead of lots of 
card marking metadata): I implemented SpurSweeper which only sweeps but works 
very well.- Currently the marking phase removes all forwarders and I would like 
incremental marking to maintain the same invariant (forwarders are always 
white).
- In general, Concurrent marking and sweeping have been implemented everywhere, 
but no concurrent compaction. For compaction you can make it selective (compact 
only part of the heap and the part which needs it the most) like I suggest and 
like in G1, which decreases considerably compaction pause time. Work on 
concurrent compaction is state of the art and not in production everywhere, see 
for example Shenandoah Garbage Collector
A. Shipilev
Pause-Less GC for Improving Java Responsiveness 
C. Gracie
And I will watch at talk on this topic tomorrow for the Android GC.- Some 
runtime, especially now with small servers being rent, are running on single 
core machines. So we need the low-pause GC to work incrementally aside from 
concurrently. So step 1 incremental GC. Step 2 concurrent marking and sweeping 
with low-pause for scavenge/compaction.
No more time right now. 

On Sun, Dec 3, 2017 at 6:33 AM, Ben Coman  wrote:


Hi Eliot, Clement,

On 7 July 2017 at 00:41, Eliot Miranda  wrote:
>
> > - Better support for large heaps (GC tuning API, incremental GC).
> > Pharo 64 bit is now able to manage large heap. However better
> > performance can be proposed by offering better settings for the
> > different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation
> scavenger that collects garbage in newly created objects (new space),
> and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation
> scavenger causes pauses of 1ms or less, and the mark-compact collector
> than causes pauses of around 200ms.  Both account for about 0.75% of
> entire execution time (1.5% total), so the scavenger is invoked frequently
> and the pauses that it creates are not noticeable.  But the occasional
> pauses created by the mark-compact collector /are/ noticeable,
> especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or
> mark-sweep-compact collector for old space that works incrementally,
> doing a little bit of work on each invocation, probably immediately after a 
> scavenge.
> It is intended to avoid the long pauses caused by the non-incremental
> mark-compact collector and make the system more suitable for games, music, 
> etc.

Reading http://www.mirandabanda.org/ cogblog/2013/09/13/lazy- 
become-and-a-partial-read- barrier/
  "An alternative implementation, oft-used in Lisp systems, is to add a
  read barrier to all object access, and mark objects as forwarders.
  This can be used to implement a >>>lazy copying garbage collection
  where objects are copied from one semi-space to another in parallel to the
  main program (the “mutator”).  To become, or move an object one replaces the
  object’s header or first field with a forwarding pointer to the desired target
  or copy in a new location, marking the “corpse” as forwarded.  The program
  checks the forwarded flag on each access.  If there is hardware support,
  as in a Lisp machine, this can work well.  But without hardware support,
  and like the object table representation, it has costs, slowing down
  program 

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

[Luke Gorrie]

Thanks Ben & Henrik for the ideas!

I will scratch my head a bit and then try to do something sensible. :-)

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

[Henrik Sperre Johansen]

For proxies/wrappers, you might want to look into how the Ghost framework
interrupts sends.
(I don't know the exact details, but I'd guess step 1 is subclassing
ProtoObject, rather than Object)

For your specific case, isn't there already a *Struct class in the image you
can subclass, create a class method containing the fields definition, and
then have accessors autogenerated, rather than using doesNotUnderstand:?
Usually, that would make the code easier to browse/debug later on, doesn't
confuse the syntax highlighter/code critic in uses of an otherwise undefined
field, etc.

Cheers,
Henry



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html