[Pharo-project] Object visitor

[Martin Dias]

Hi,

We want to do some processing over the graph of object references.

Do you recommend an implementation of visitor pattern to iterate object
instance variables?

Best regards,
Martin and Tristan
___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Re: [Pharo-project] Object visitor

[Martin Dias]

On Thu, Sep 2, 2010 at 4:53 PM, Alexandre Bergel wrote:

> Yes.
> Visitor is a convenient pattern to perform a computation over each element
> of a recursive data structure.
> This pattern works well for trees. You need to pay attention to cycle when
> using it on a graph. This might be expensive if you have a large graph and
> you use a set to keep track of the element you run over.
>
>
It has to be as faster as possible, but I think that the Set would be ok.




> Do you want to do this on any object of the image?
>
>
Yes, for any object. It's for serializing a graph of objects into a
stream. We need to iterate over the transitive closure of object references.

Do you know an existing package that implements this pattern?





> Alexandre
>
>
>
Thank you for the answer
Martin




> On 2 Sep 2010, at 10:36, Martin Dias wrote:
>
> > Hi,
> >
> > We want to do some processing over the graph of object references.
> >
> > Do you recommend an implementation of visitor pattern to iterate object
> instance variables?
> >
> > Best regards,
> > Martin and Tristan
> > ___
> > Pharo-project mailing list
> > Pharo-project@lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
> --
> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> Alexandre Bergel  http://www.bergel.eu
> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
>
>
>
>
>
>
> ___
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

[Pharo-project] Automatic testing in different images

[Martin Dias]

Hello,

I'm working on a prototype for binary serialization of objects, and I would
like to do tests and benchmarks for serializing on a stream in one image and
de-serialize them in another, maybe with different conditions (other
version, VM, etc).

Could you recommend me how to do that in an automatic way?

Thanks
Martin
___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Re: [Pharo-project] Automatic testing in different images

[Martin Dias]

I will look at Magma, and I think we will use Hudson later.

Thank you

Martin

2010/9/28 Göran Krampe 

> On 09/28/2010 12:44 PM, Simon Denier wrote:
>
>>
>> On 28 sept. 2010, at 11:24, Martin Dias wrote:
>>
>>  Hello,
>>>
>>> I'm working on a prototype for binary serialization of objects, and I
>>> would like to do tests and benchmarks for serializing on a stream in one
>>> image and de-serialize them in another, maybe with different conditions
>>> (other version, VM, etc).
>>>
>>> Could you recommend me how to do that in an automatic way?
>>>
>>
>>
>> Hudson seems a good way to go
>>
>
> Or take a look at the test framework in Magma, it has orchestrated tests
> between multiple images, probably what you want.
>
> And oh, Magma also has of course binary serialization stuff in it - so
> worth looking at from that perspective too.
>
> regards, Göran
>
>
> ___
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

[Pharo-project] Faster access for refactoring commands

[Martin Dias]

Hi,

I really like to use refactoring commands in OmniBrowser, but I would like
to access them faster. I think that it would be nice to have:

- A shortcut that popups a menu with the refactor commands only.
- A unique shortcut for rename (which target is the current selection:
categories, class, method, class variable, instance variable, temporary)

What do you think about it?
Do you know if there is something like that already available to use?

Martin
___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Re: [Pharo-project] Faster access for refactoring commands

[Martin Dias]

Ok, I will see how does it work.



On Wed, Oct 20, 2010 at 3:46 PM, Stéphane Ducasse
 wrote:
>
> On Oct 20, 2010, at 1:32 PM, Martin Dias wrote:
>
>> Hi,
>>
>> I really like to use refactoring commands in OmniBrowser, but I would like 
>> to access them faster. I think that it would be nice to have:
>>
>> - A shortcut that popups a menu with the refactor commands only.
>> - A unique shortcut for rename (which target is the current selection: 
>> categories, class, method, class variable, instance variable, temporary)
>>
>> What do you think about it?
>> Do you know if there is something like that already available to use?
>
> Yes I would love to have a better organization with the most used refactoring 
> actions also on the top of the menu.
> I know lukas does not like that. Now what we can do it to have a package to 
> add these items on top of
> OB and we load it, after OB-RB and everybody is happy.
>
> Can you provide that code?
>
>
>> Martin
>> ___
>> Pharo-project mailing list
>> Pharo-project@lists.gforge.inria.fr
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> ___
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

___
Pharo-project mailing list
Pharo-project@lists.gforge.inria.fr
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

[Pharo-project] MC2 on Pharo

[Martin Dias]

Hi, is possible to load and run Monticello2 in Pharo?

Thank you,
Martin

[Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi all

Last months I and Tristan have been working on Fuel project, an object
binary serialization tool. The idea is that objects are much more
times loaded than stored, therefore it is worth to spend time while
storing in order to have faster loading and user experience. We
present an implementation of a pickle format that is based on
clustering similar objects.

There is a summary of the project below, but more complete information
is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel

The implementation still needs a lot of work to be really useful,
optimizations should be done, but we'll be glad to get feedback of the
community.


= Pickle format =

The pickle format and the serialization algorithm main idea, is
explained in this slides:

http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example


= Current features =

- Class shape changing (when a variable has been added, or removed, or
its index changed)
- Serialize most of the basic objects.
- Serialize (almost) any CompiledMethod
- Detection of global or class variables
- Support for cyclic object graphs
- Tests


= Next steps =

- Improve version checking.
- Optimize performance.
- Serialize more kinds of objects:
-- Class with its complete description.
-- Method contexts
-- Active block closures
-- Continuation
- Some improvements for the user:
-- pre and post actions to be executed.
-- easily say 'this object is singleton'.
- Partial loading of a stored graph.
- Fast statistics/brief info extraction of a stored graph.
- ConfigurationOfFuel.
- Be able to deploy materialization behavior only (independent from
the serialization behavior)


= Download =

In a Pharo 1.1 or 1.1.1 evaluate:

Gofer new
squeaksource: 'Fuel';
version: 'Fuel-MartinDias.74';
version: 'FuelBenchmarks-MartinDias.4';
load.


= Benchmarks =

You can run benchmarks executing this line (results in Transcript):

FLBenchmarks newBasic run.


Thank you!
Martin Dias

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi Tudor,

Thank you.
Yes, just to test the project in a real case, I created a package that
uses Fame metadescription like MSE to export and import moose models.
In the main page I talk a bit about the extension, but I didn't
mentioned it in the summary mail. It needs to be better tested... and
a progress bar like MSE!

In Moose 4.1:

Gofer new
squeaksource: 'Fuel';
version: 'Fuel-MartinDias.74';
version: 'FuelFameExtension-MartinDias.10';
load.

And opening a new MoosePanel it will have the export to FL and import
from FL options.

I would like to have your feedback!

Martin


On Thu, Dec 9, 2010 at 10:02 AM, Tudor Girba  wrote:
> Hi Martin,
>
> Nice project.
>
> I noticed that you have a package FuelFameExtension. Is this done for the 
> Fame meta engine? If yes, I would be interested in testing it, especially 
> that in the context of Moose we do load the objects significantly more often 
> than we store them :).
>
> Cheers,
> Doru
>
>
> On 8 Dec 2010, at 17:50, Martin Dias wrote:
>
>> Hi all
>>
>> Last months I and Tristan have been working on Fuel project, an object
>> binary serialization tool. The idea is that objects are much more
>> times loaded than stored, therefore it is worth to spend time while
>> storing in order to have faster loading and user experience. We
>> present an implementation of a pickle format that is based on
>> clustering similar objects.
>>
>> There is a summary of the project below, but more complete information
>> is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel
>>
>> The implementation still needs a lot of work to be really useful,
>> optimizations should be done, but we'll be glad to get feedback of the
>> community.
>>
>>
>> = Pickle format =
>>
>> The pickle format and the serialization algorithm main idea, is
>> explained in this slides:
>>
>> http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example
>>
>>
>> = Current features =
>>
>> - Class shape changing (when a variable has been added, or removed, or
>> its index changed)
>> - Serialize most of the basic objects.
>> - Serialize (almost) any CompiledMethod
>> - Detection of global or class variables
>> - Support for cyclic object graphs
>> - Tests
>>
>>
>> = Next steps =
>>
>> - Improve version checking.
>> - Optimize performance.
>> - Serialize more kinds of objects:
>> -- Class with its complete description.
>> -- Method contexts
>> -- Active block closures
>> -- Continuation
>> - Some improvements for the user:
>> -- pre and post actions to be executed.
>> -- easily say 'this object is singleton'.
>> - Partial loading of a stored graph.
>> - Fast statistics/brief info extraction of a stored graph.
>> - ConfigurationOfFuel.
>> - Be able to deploy materialization behavior only (independent from
>> the serialization behavior)
>>
>>
>> = Download =
>>
>> In a Pharo 1.1 or 1.1.1 evaluate:
>>
>> Gofer new
>>       squeaksource: 'Fuel';
>>       version: 'Fuel-MartinDias.74';
>>       version: 'FuelBenchmarks-MartinDias.4';
>>       load.
>>
>>
>> = Benchmarks =
>>
>> You can run benchmarks executing this line (results in Transcript):
>>
>> FLBenchmarks newBasic run.
>>
>>
>> Thank you!
>> Martin Dias
>>
>
> --
> www.tudorgirba.com
>
> "Sometimes the best solution is not the best solution."
>
>
>

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi all,

Thank you for the discussion, is very interesting for me.

Thanks Adrian, I don't have benchmarks with ImageSegment, so I like to
see the numbers, how could I reproduce that benchmarks? it would be
useful for me.

I think that ImageSegment is the best if fits the needs. Maybe if in a
future we implement using primitives some things it could be more
comparable. As someones said, Fuel is more similar to Parcels, in fact
we started working based on this paper:
http://scg.unibe.ch/archive/papers/Mira05aParcels.pdf

About if "objects are much more times loaded than stored", yes... Fuel
is not the universal serialization solution, but I believe it could be
useful in some cases.

Beside of the speed, I think that is also important the "user
experience" part of the purpose ("is worth to spend time while storing
in order to have faster loading and user experience"), in the sense of
giving more features that makes easier the life of the developer to
share their object structures.

Maybe Fuel could give more flexibility than ImageSegment to select
which objects of the graph store and which not. It could let the user
define custom rules.

For example if FuelFameExtension is installed, when Fuel stores an
object first looks at the Fame metadescription of that object and then
only stores the "not derived attributes" of that object. That was
necessary to export and import properly Moose models.

Martin



On Thu, Dec 9, 2010 at 3:58 PM, Igor Stasenko  wrote:
> 2010/12/9 Levente Uzonyi :
>> On Thu, 9 Dec 2010, Stéphane Ducasse wrote:
>>
>>> BTW
>>> when giving feedback consider that the guy doing that is spending a lot of
>>> time and this will be his master
>>> and that the code was not optimize and that there is no dedicated
>>> primitive in play.
>>>
>>> So we will see at the end and I was thinking that our little community
>>> would be much more positive but
>>> we will continue because we believe that there is some value in that.
>>
>> Don't get me wrong, I'm not saying that Fuel is not useful. I'm saying that
>> improving code loading performance is not that important.
>>
>
> Sort of.
> But what is most important, i think that you can exchange objects
> between images.
> MC really allows you to exchange only with source code,
> while with Fuel, i think you could put any object/data into binary
> package, and don't bother with
> inventing the pervasive ways how to recreate complex (or big) data
> structures from array literals :)
>
> Another interesting aspect of binary format is that you can give
> binary to people without
> disclosing source code.. (waving to corporate world ;)
>
>>
>> Levente
>>
>>>
>>> Stef
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi Colin

Another thing to consider are the clusters. I think they are good
because they avoid a lot of overhead. Following the example used in
the slides, for serializing N rectangles, it writes:

- in vertexes section:
'Rectangle'
'origin'
'corner'
N

- in edges section: 2*N indexes (references to the origin and corner points)


Without clustering, I think each rectangle stored should have some
header to say that its an instance of Rectangle.

Martin


On Fri, Dec 10, 2010 at 11:03 AM, Tristan Bourgois
 wrote:
>
>> Hi Martin,
>>
>> Looks very interesting. Reading the material you posted, the question
>> that jumps out at me is this: from the benchmarks, it's clear that
>> materializing is faster than serializing, but it's not clear why. How
>> does separating the nodes from the edges of the object graph make
>> materializing faster?
>>
>> Thanks,
>>
>> Colin
>
> Hi!
>
> Separate the nodes from the edges of the object permit during the
> serialization
> to pass only one time on the byte code.
> So first we pass on the node so we recreate the objects/nodes and second
> when
> we pass on the edges we recreate the links between the objects.
>
> I hope I answered your question :)
>
> Tristan
>

Re: [Pharo-project] ProgressDisplay package released

[Martin Dias]

Hi Ralph,
I tried to load ProgressDisplay package on Pharo 1.1.1. It depends on 2
nonexistent classes: Generator and InputSensor. How can I obtain them?

Thanks,
Martin


On Mon, Aug 2, 2010 at 5:57 AM, Serge Stinckwich  wrote:

> Yes sell you work ! Not everyone has time to test ;-)
>
> On Mon, Aug 2, 2010 at 3:58 PM, Alexandre Bergel 
> wrote:
> > Yep, a screenshot will be cool
> >
> > Alexandre
> >
> >
> > On 2 Aug 2010, at 10:52, Stéphane Ducasse wrote:
> >
> >> Cool
> >>
> >> Can you tell us some of the added features?
> >>
> >> Stef
> >>
> >>> I recently released the first draft of my package ProgressDisplay to
> >>> SqueakSource.
> >>> It is like SystemProgressMorph et. al. but with a lot of bells and
> >>> whistles added.
> >>> It is not a suitable replacement for SystemProgressMorph though because
> it is
> >>> too large and specific for that. SystemProgressMorph could be made a
> subclass
> >>> of one of its classes though.
> >>>
> >>> It is designed largely to meet my specific needs but since it's MIT
> anyone may
> >>> use it, steal components, or move components of it to Squeak/Pharo.
> >>> It has a utility class category with useful classes for things like
> >>> array permutation generation and array multiset permutation
> >>> generation, numeric partitioning of a number, and a class RangeList
> >>> (which I will be expanding upon later).  All of these classes will be
> >>> removed when the package meant to contain them is ready (not soon).
> >>> There are also a number of utility methods that could be stolen or put
> >>> into Squeak/Pharo proper.
> >>>
> >>> It probably has lots of bugs, so bug reports appreciated.
> >>> Comments on its function, design, and implementation most welcome.
> >>> Proposals for improvements or extensions will be considered but no
> promises.
> >>> If you need access to make changes let me know.
> >>> If Levente Uzonyi likes it but feels compelled to rewrite it
> >>> completely from scratch, thats okay too. :-)
> >>>
> >>> ONE OF MY REASONS FOR RELEASING IT IS BECAUSE OF A
> >>> BUG I COULD NOT FIX.
> >>> It is supposed to be possible to turn on the ability to catch user
> interrupts
> >>> but when this feature is turned on and an interrupt generated while
> >>> the progress bar is displayed the interrupt is caught but not processed
> >>> until the progress bar closes. :-(
> >>> Of course the interrupt was meant to be processed immediately.
> >>> If anyone can tell me how to fix this bug (or better still send me the
> >>> code or add the code to the package) that would be great!
> >>>
> >>> Regards,
> >>>
> >>> Ralph Boland
> >>>
> >>> --
> >>> Quantum Theory cannot save us from the tyranny of a deterministic
> universe
> >>> but it does give God something to do
> >>>
> >>> ___
> >>> Pharo-project mailing list
> >>> Pharo-project@lists.gforge.inria.fr
> >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >>
> >>
> >> ___
> >> Pharo-project mailing list
> >> Pharo-project@lists.gforge.inria.fr
> >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
> > --
> > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> > Alexandre Bergel  http://www.bergel.eu
> > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
> >
> >
> >
> >
> >
> >
> > ___
> > Pharo-project mailing list
> > Pharo-project@lists.gforge.inria.fr
> > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
> >
>
>
>
> --
> Serge Stinckwich
> UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam
> Every DSL ends up being Smalltalk
> http://doesnotunderstand.org/
>
> ___
> Pharo-project mailing list
> Pharo-project@lists.gforge.inria.fr
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>

[Pharo-project] Progress bar design pattern

[Martin Dias]

Hello

I have a couple of algorithms and I want to show the progress while they
run. I played with the progress bar and it's okay for my needs.

The progress bar should be pluggable and decoupled of the algorithms.

I am writing to you to ask about good designs for my problem. I hope I
haven't expressed the problem in a too abstract way.

The design I have in mind is a kind of observer pattern: the serialization
algorithm publishes information about the run; a specific listener
implements the progress bar for that serialization algorithm, interpreting
the information published.

Thanks,
Martin

Re: [Pharo-project] Progress bar design pattern

[Martin Dias]

Hi!

On Wed, Mar 30, 2011 at 2:13 AM, Daniel Lyons wrote:

>
> On Mar 29, 2011, at 6:17 PM, Martin Dias wrote:
>
> I have a couple of algorithms and I want to show the progress while they
> run. I played with the progress bar and it's okay for my needs.
>
> The progress bar should be pluggable and decoupled of the algorithms.
>
> I am writing to you to ask about good designs for my problem. I hope I
> haven't expressed the problem in a too abstract way.
>
> The design I have in mind is a kind of observer pattern: the serialization
> algorithm publishes information about the run; a specific listener
> implements the progress bar for that serialization algorithm, interpreting
> the information published.
>
>
> I'm a newcomer here, so I'm sharing my experience with non-Smalltalk
> systems and my best guess what would constitute good designs. I hope other
> real Smalltalkers will poke some holes in this and point out the right way
> to do things! ;)
>
> The simplest design, IMO, would be this API:
>
> MyProgressBar>>percentComplete: aNumber
> MyProgressBar>>beIndeterminant
> MyProgressBar>>beDeterminant
>

Sorry, what is an indeterminant progress bar?


>
> You would manually invoke percentComplete: with a new percent complete each
> time you want to advance the progress bar. I've seen this kind of thing in
> practice somewhere. Maybe it was Cocoa?
>
> The next level of abstraction would be to go a bit more jQuery UI and
> remove percentComplete: and use this instead:
>
> MyProgressBar>>maximum: anInteger
> MyProgressBar>>current: anInteger
>
> Now you'll calculate the progress bar based on some quantity of "tasks" you
> want to do. You could even go further and just have:
>
> MyProgressBar>>incrementProgress
>
> instead of letting your clients tell you which one they're on. That would
> prevent progress bar relapse. Either of these would be easy to implement on
> top of the above API using percent.
>
> The simplest observer-type pattern would be to rely on the built-in
> observer/event system and use the stuff from Object's updating protocol or
> events-registering or events-triggering. I don't know which one of these is
> modern or preferred. Anybody? :) At any rate, they make it possible to
> expose a simple object/message protocol as in Polymorph, where your progress
> bar widget would be listening for change notifications from your domain
> object. Then you can ask the same questions about your model as above, in
> terms of whether you want to work in terms of simply percent complete or N/M
> or what.
>
>
I was looking to use String>>displayProgressAt:from:to:during:. The example
shown on its comment is:

'Now here''s some Real Progress'
displayProgressAt: Sensor cursorPoint
from: 0 to: 10
during: [:bar |
1 to: 10 do: [:x | bar value: x.
(Delay forMilliseconds: 500) wait]].

But my doubt was how to connect the algorithm progress with that "bar value:
x".

I would like to avoid that the model (my serialization algorithm) explicitly
speak of percentages, I prefer the calculus of the percentage to be done in
another place, some observer that interprets the progress and then updates
the bar.

I will try to follow the idea I understood of your last paragraph:

I will have the algorithm run reified in a class, let say MyAlgorithmRun,
which sends itself #changed each time it has progressed, and a dependent,
let say an instance of MyAlgorithmRunProgressObserver, receives an #update:.
Then the observer interprets the current state of the run, and sends #value:
to the UI progress bar.

Is it okay?



> Another question is whether or not you want to provide a string message
> status to go under the progress bar.
>
> Does this help at all, or did I miss the point?
>
>
Yes! for me it's very useful to think, thanks!


> —
> Daniel Lyons
>
>

Re: [Pharo-project] Progress bar design pattern

[Martin Dias]

On Wed, Mar 30, 2011 at 6:12 AM, Henrik Johansen <
henrik.s.johan...@veloxit.no> wrote:

>
> On Mar 30, 2011, at 7:13 23AM, Daniel Lyons wrote:
>
>
> On Mar 29, 2011, at 6:17 PM, Martin Dias wrote:
>
> I have a couple of algorithms and I want to show the progress while they
> run. I played with the progress bar and it's okay for my needs.
>
> The progress bar should be pluggable and decoupled of the algorithms.
>
> I am writing to you to ask about good designs for my problem. I hope I
> haven't expressed the problem in a too abstract way.
>
> The design I have in mind is a kind of observer pattern: the serialization
> algorithm publishes information about the run; a specific listener
> implements the progress bar for that serialization algorithm, interpreting
> the information published.
>
>
> I'm a newcomer here, so I'm sharing my experience with non-Smalltalk
> systems and my best guess what would constitute good designs. I hope other
> real Smalltalkers will poke some holes in this and point out the right way
> to do things! ;)
>
> The simplest design, IMO, would be this API:
>
> MyProgressBar>>percentComplete: aNumber
> MyProgressBar>>beIndeterminant
> MyProgressBar>>beDeterminant
>
> You would manually invoke percentComplete: with a new percent complete each
> time you want to advance the progress bar. I've seen this kind of thing in
> practice somewhere. Maybe it was Cocoa?
>
> The next level of abstraction would be to go a bit more jQuery UI and
> remove percentComplete: and use this instead:
>
> MyProgressBar>>maximum: anInteger
> MyProgressBar>>current: anInteger
>
> Now you'll calculate the progress bar based on some quantity of "tasks" you
> want to do. You could even go further and just have:
>
> MyProgressBar>>incrementProgress
>
> instead of letting your clients tell you which one they're on. That would
> prevent progress bar relapse. Either of these would be easy to implement on
> top of the above API using percent.
>
> The simplest observer-type pattern would be to rely on the built-in
> observer/event system and use the stuff from Object's updating protocol or
> events-registering or events-triggering. I don't know which one of these is
> modern or preferred. Anybody? :) At any rate, they make it possible to
> expose a simple object/message protocol as in Polymorph, where your progress
> bar widget would be listening for change notifications from your domain
> object. Then you can ask the same questions about your model as above, in
> terms of whether you want to work in terms of simply percent complete or N/M
> or what.
>
> Another question is whether or not you want to provide a string message
> status to go under the progress bar.
>
> Does this help at all, or did I miss the point?
>
> —
> Daniel Lyons
>
> I've used Announcements for this in the past, and liked it quite alot.
>
> I will take a look on that.


> - Worker announces WorkCompleted or some such.
> Announcement holds details of the work it's done, in my case different
> properties of an analysis like leak rate, .
>
> - GUI/Client is set up before analysis starts, and initialized with the
> total work to be done.
> Annnouncement handling contains code to estimate how much progress the Work
> indicated by it was of the overall total.
>
> I also used a different client which provided ETA based on a more
> sophisticated analysis of the Work announcements.
>

Wow! I don't need to estimate time but it's very interesting think on that.


>
> Needless to say, there's no hard dependency on a UI in such a scheme, as
> there simply won't be any clients instantiated if you run headless. (well,
> unless you make one writing to stdout or something)
>
> Not sure what would be reasonable to put in a Work-annnouncement intended
> as superclass for custom solutions...
> In other words, the downside is that this approach doesn't lead to
> something you can use out-of-the-box, you always have to do some
> customization, either/both to what data you provide, or/and to how you give
> feedback.
> (In this case, a  standard widget using the simplest design you described
> is preferrable, btw, the pluggability is done at a different level than the
> widget)
>
> In this moment I am interested in a simple solution, I will not write a
framework, but I like the discussion. Thanks!


> Cheers,
> Henry
>

Re: [Pharo-project] Progress bar design pattern

[Martin Dias]

On Thu, Mar 31, 2011 at 5:28 PM, Daniel Lyons wrote:

> On Thu, Mar 31, 2011 at 04:58:04PM -0300, Martin Dias wrote:
> > > The simplest design, IMO, would be this API:
> > >
> > > MyProgressBar>>percentComplete: aNumber
> > > MyProgressBar>>beIndeterminant
> > > MyProgressBar>>beDeterminant
> > >
> >
> > Sorry, what is an indeterminant progress bar?
>
> On your Mac, it's the barber pole; on Windows, it's when the progress
> bar just sort of throbs. It's for when you initially are calculating
> how long the task is going to take, or for cleanup after the task is
> complete.
>
>
Ah! I need that. During big part of the run I don't know the size of the
task.


> --
> Daniel
>
>

Re: [Pharo-project] Basic tricks for improving a serializer?

[Martin Dias]

On Tue, May 17, 2011 at 7:16 PM, Igor Stasenko  wrote:

> On 17 May 2011 22:58, Mariano Martinez Peck  wrote:
> >
> >
> > On Tue, May 17, 2011 at 10:31 PM, Sven Van Caekenberghe 
> > wrote:
> >>
> >> On 17 May 2011, at 21:57, Mariano Martinez Peck wrote:
> >>
> >> > Sven, I want to make it work :)
> >> >
> >> > sothe missing methods I told you that I need are:
> >> >
> >> > #nextStringPut:
> >> > #nextNumber:put:
> >> > #nextInt32Put:
> >> > #nextWordPut:
> >>
> >> I guess these are pretty easy. But I think they clutter the interface of
> >> ZnBufferedWriteStream, so maybe you should make a subclass.
> >>
> >
> > Yeah, don't worry. I can even duplicate the class hehehe
> >
> >>
> >> > #contents
> >> >
> >> > Implement #contents I guess it is something like:
> >> >
> >> > ZnBufferedWriteStream >> contents
> >> > ^ stream contents
> >>
> >> Why to you need #contents ?
> >
> > becasue I am an idiot. No, I don't need it. You are correct. Thanks for
> > asking.
> >
> >>
> >> I would say that it goes a bit against the concept of a stream as a sink
> >> of data.
> >> I haven't looked, but I would guess that saying #contents to a
> FileStream
> >> is not efficient.
> >>
> >> > Those missing methods I need are implemented PositionableStream. I
> took
> >> > the implementation from there and  put it in ZnBufferedWriteStream.
> >> > I just added to them a first line "self flushBufferIfFull."
> >>
> >> That is probably OK, except when your string becomes larger than the
> >> buffer. Have a look at #nextPutAll:
> >>
> >
> > I am not sure if I understood. The following are correct for sure then:
> >  #nextNumber:put:
> >  #nextInt32Put:
> >  #nextWordPut:
> >
> >
> > And #nextStringPut:   is like this:
> >
> > nextStringPut: s
> > "Append the string, s, to the receiver.  Only used by DataStream.
> Max
> > size of 64*256*256*256."
> >
> > | length |
> > self flushBufferIfFull.
> > (length := s size) < 192
> > ifTrue: [self nextPut: length]
> > ifFalse:
> > [self nextPut: (length digitAt: 4)+192.
> > self nextPut: (length digitAt: 3).
> > self nextPut: (length digitAt: 2).
> > self nextPut: (length digitAt: 1)].
> > self nextPutAll: s asByteArray.
> > ^s
> >
>
> Sorry, but i can't resist commenting on that.
> Why, if you demand from stream to implement #nextInt32Put:
> a the same time, you writing code like this
>
> self nextPut: (length digitAt: 4)+192.
> self nextPut: (length digitAt: 3).
> self nextPut: (length digitAt: 2).
> self nextPut: (length digitAt: 1)
>
> ?
> Then just extend your serializer with a notion of 'length' field,
> which you can use for anything where you need to encode length/size value,
> but not just for Strings.
> So, then the above method could be as short as:
>
> nextStringPut: s
>  self putLength: s size.
>  self nexPutAll: s asByteArray.
>
> and here you have a potential caveat because your string could be
> WideString .. muhahaha.
>
> So, i suggest you to reconsider the way how you serializing strings.
> Instead what you could do is to extend ByteString and WideString (and
> perhaps similarily do for ByteSymbol and WideSymbol),
> the methods which is responsible to turning a receiver in a sequence
> of bytes, and then simply put it into output stream,
> whatever it might be.
>
> Then you don't need #nextStringPut: because its
> a) not polymorphic, because apparently serializing ByteString should
> be different from serializing WideString
> b) instead you implementing this in
> Byte/WideString>>serializeToFuelStream: aStream and you done.
>

Yes, I am not sure if #nextStringPut: is not polymorphic, but I think that
with #serializeToFuelStream: we can avoid converting the WideString to
ByteArray just to write the ByteArray to the stream, and instead just write
the WideString to the stream. Right?



> >
> > which will finally send #nextPutAll:  that does correct check.
> >
> >
> >>
> >> > The way I am creating it is like this:
> >> >
> >> > stream := ZnBufferedWriteStream on: ((FileDirectory default
> >> > forceNewFileNamed: 'mariano') binary).
> >> >
> >> > So...is this correct Sven?   My tests are green :)
> >>
> >> That seems OK
> >
> > Ok.
> >
> >>
> >> > Still need to try benchmarks.
> >>
> >> You might also consider playing with making the buffer larger.
> >>
> >
> > yeah, I was trying that. Just by curious, why you put a 2^X number there?
> >
> > Ahh and for reading we don't need something special because it is already
> > buffered? (I have been told that)
> >
> > Thanks a lot Sven,
> >
> > --
> > Mariano
> > http://marianopeck.wordpress.com
> >
> >
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

[Pharo-project] ESUG SummerTalk - Fuel, binary object serializer

[Martin Dias]

Hi folks. I am really happy to announce that ESUG is sponsoring me for Fuel
development through the ESUG SummerTalk. I am Martin Dias, a student at
Buenos Aires, Argentina. The idea behind this SummerTalk is to implement
Fuel, a binary, fast and general-purpose object graph serializer in
Pharo<http://www.pharo-project.org/>.
It is based on VisualWorks' Parcels ideas.

Actually, the project has already started since several months. Tristan
Bourgois and I started with the project while doing an internship with RMoD,
INRIA <http://rmod.lille.inria.fr/web/pier>. Since a couple of months, Mariano
Martinez Peck <http://marianopeck.wordpress.com/> joined the team, and now
he is the official mentor in the SummerTalk.

ESUG website for SummertTalk:
http://www.esug.org/wiki/pier/Promotion/SummerTalk/SummerTalk2011

The website with all the necessary information is here:
http://rmod.lille.inria.fr/web/pier/software/Fuel
It even includes slides explaining the algorithm. In addition, a paper is in
progress.

For the moment, Fuel already provides the following features:

- Fast pickle format. It is much faster to materialize than to serialize.
- Correctly support class reshape (when the class of serialized objects has
changed).
- Serialize ANY kind of object. For the moment there is no object to our
knowledge that we cannot serialize and materialize.
- Be able to completely serialize classes and traits (not just a global
name).
- Support cycles and avoid duplicates in the graph.
- Integration to Moose <http://www.moosetechnology.org/> with an extension
to export and import their models.
- Detection of globals: for example if you serialize Transcript, it is not
duplicated and instead managed as a global reference.
- Solve common problems like Set rehash.
- Buffered writing: we use a buffered write stream for the serialization
part (thanks Sven!).
- No need of special support from the VM.
- Try to have a good object oriented design.
- Well tested (about 120 tests, for the moment).
- Large set of benchmarks (even benchmarks for Moose extension).

And of course, there are a lot features for the future. You can see some of
them in the website and some in the issue tracker:
http://code.google.com/p/fuel/issues/list

We really appreciate all kind of feedback and comments. If you want to try
it, check in the website how to do it. It is extremely easy.

Once again, I want to thank a lot to ESUG for sponsoring the project. I plan
to create a "news" section in the website with some RSS. I will keep you
informed.

Best regards,
Martin

Re: [Pharo-project] [Seaside] Re: Re: ESUG SummerTalk - Fuel, binary object serializer

[Martin Dias]

Hi Yanni,

On Wed, May 25, 2011 at 1:10 PM, Yanni Chiu  wrote:

> On 25/05/11 11:53 AM, Mariano Martinez Peck wrote:
>
>>
>> Sorry Yanni, I didn't follow. Could you please explain a bit more? what
>> do you want to serialize? do you want to be able to choose some classes
>> as light and some as non-light? where do you want to materialize ? in
>> the same image or in another one ?   When you said discardwhat would
>> you do with the instances of those non-light classes for example? you
>> don't materialize them? and what happens to the objects that were
>> pointing to them ?  why would be the scenario useful for ? security ?
>>
>
> 
> Yes, security. Here's my first post again, with different formatting:
>
> In another use case, I'd like to serialize from one image, and deserialize
> in another image - *under end user control*. [e.g. web app]
>
>
> The issue here is that "nasty" code could be introduced:
> - capture the Fuel output
> - deserialize, add nasty code, re-serialize
> - then send onward for import to image.
>
> Would it be possible to have some sort of "virus" filter?
> 
>
> So a simple "safe-mode" option on de-serialization would probably be
> sufficient.


It is a good point.

For the moment, when you deserialize a full class into the image, their
methods are created, and bytecodes are copied from the stream without any
validation check.

Anyway, you could deserialize the class, run your own validations, and then
install the class (I mean, add the class to Smalltalk globals, do the class
initialization, run announcements).



>
>
> ___
> seaside mailing list
> seas...@lists.squeakfoundation.org
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi Eliot,

I am glad to receive your feedback. I definitely agree with that, your
changes have been added on last version.

The reason for that cached information was mainly to support a Mariano's
project requirement: he needs to frequently do lots of small serializations.
But in any case, that cache should be managed externally of Fuel.

In my computer, this bench:
[ FLGlobalMapper newWith: nil ] bench.

went from '402,000 per second' to '35.8 per second'. But in the common use
case of a serializer it is really imperceptible.

Thank you very much.
Martin


On Wed, Jun 8, 2011 at 5:20 PM, Eliot Miranda wrote:

>
>
> On Wed, Jun 8, 2011 at 11:59 AM, Eliot Miranda wrote:
>
>> Hi Mariano and Martin,
>>
>> some feedback.  The ClassPoolBindings cache
>> in FLClassPoolAssociationMapper feels like a mistake to me.  I would just
>> compute this map at the start of serialization.  It'll take hardly any time.
>>  Whereas there's lots of scope for this information, if cached, to become
>> stale over time.
>>
>> Find attached.  If you're happy with this then before or after remember to
>> evaluate
>> SystemChangeNotifier uniqueInstance noMoreNotificationsFor:
>> FLClassPoolAssociationMapper
>>
>
> and the same goes for FLGlobalMapper.  Also remember SystemChangeNotifier
> uniqueInstance noMoreNotificationsFor: FLGlobalMapper
>
>
>>
>> best
>> Eliot
>>
>
>

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

On Wed, Jun 8, 2011 at 10:35 PM, Eliot Miranda wrote:

> Hi Martin and Mariano,
>
> a couple of questions.  What's the right way to exclude certain objects
> from the serialization?  Is there a way of excluding certain inst vars from
> certain objects?
>
>
Hi,

We are close to have this feature, but still is an open issue.

The idea is to to support this "statically" via:

ClassWithTransients >> fuelTransientInstanceVariables
   ^ #(myTransientValue1 myTransientValue1)

or more dinamically via:

FLSerializationSettings newDefault
onClass: ClassWithTransients
declareTransients: #(myTransientValue1 myTransientValue1).

Do you have a better idea?

Cheers
Martin



> best
> Eliot
>

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi

> Has anyone done any serious work with Paul Baumann's SRP?
> >
> > http://sourceforge.net/projects/srp/
>
> And also with MessagePack (http://msgpack.org/), a highly space/time
> efficient binary serialization protocol. There is an implementation on
> SqueakSource at http://www.squeaksource.com/MessagePack and a
> high-level object-serialization protocol on top of that at
> http://www.squeaksource.com/STOMP.
>
>
I have never used SRP or MessagePack, both seems great. An important
difference of Fuel is that it is not focused in cross-dialect or
cross-language object exchange.

Re: [Pharo-project] Fuel version 1.4 uses #isBinary, and SIXX uses #tab:

[Martin Dias]

On Mon, Jun 13, 2011 at 7:00 PM, Mariano Martinez Peck <
marianop...@gmail.com> wrote:

>
>
> On Mon, Jun 13, 2011 at 11:52 PM, Yanni Chiu  wrote:
>
>> FLBufferedWriteStream>>sizeBuffer: calls #isBinary, which is not currently
>> implemented by FSWriteStream - probably because #isBinary is not specified
>> by ANSI. Is there a reason why the stream is not always binary?
>>
>>
> I think no and I think you are correct. Fuel is binary, and
> FLBufferedWriteStream is only for Fuel, so we remove that #isBinary. Martin,
> do you agree?
>

Yes, I agree. You know more than me about the buffered stream.

The only objection I see is the open issue we have about supporting an
alternative text format in Fuel...

How can I put a String in a FSWriteStream?


>
>
>> Also, when using SIXX with FSWriteStream, I had to add the non-ANSI #tab:
>> method to FSWriteStream. There's probably a dozen calls to #tab: in the SIXX
>> code. Should #tab: be added to FSWriteStream somehow? Maybe a non-ANSI
>> streams extension package?
>>
>>
>>
> Esteban Lorenzano wanted also to use Colin's FileSystem in Fuel instead of
> the regular streams and that's why we have created the package
> FuelFileSystem.  So you can add all the methods you want to FSWriteStream
> under a category *FuelFileSystem. I can give you commit access to Fuel repo
> if you want.
>
> I still don't understand why you cannot directly serialize/materialize the
> PRKernel instance.
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

Re: [Pharo-project] Fuel serialization - feature/bug

[Martin Dias]

Hi Yanni

On Tue, Jun 14, 2011 at 3:12 AM, Yanni Chiu  wrote:

> On 13/06/11 6:56 PM, Yanni Chiu wrote:
>
>> On 13/06/11 5:49 PM, Mariano Martinez Peck wrote:
>>
>>>
>>> mm so you are using a stream to serialize and such stream is also
>>> being refenced from the object to serialize?
>>>
>>
>> Yes, that's it - except that the stream is an FSWriteStream on a
>> FSMemoryStore "file", which I think is an issue here. The PRKernel is
>> holding an instance of PRFuelPersistency, which holds an FSReference.
>> That FSReference happens to refer to a memory-based "file".
>>
>
> The code is at http://ss3.gemstone.com/ss/pierfuel.html
>
> You'll first need a Pier image, with the following loaded:
>
> (ConfigurationOfFilesystem project version:  '2.0.1-baseline') load
>
> (ConfigurationOfFuel project version:  '1.4') load
>  OR
> (ConfigurationOfFuel project version:  '1.5-baseline') load
>
> If you run PRFuelPersistencyTest, then you'll see the stack trace. I saw
> the same problem with 1.4 and with 1.5-baseline.
>
> In the 1.5-baseline, I added #fuelIgnoredInstanceVariableNames to
> PRFuelPersistency and FSMemoryStore (to ignore 'directory' and 'root',
> respectively) but the same problem remained. So I think it's a bug.
>

I didn't try in the Pier image, but I hope your problem is fixed with the
last version (Fuel-MartinDias.259).

serialize the stream where you are serializing is a test case I had never
thought!

BTW, is it fine to get nil on these variables? I mean, is the FSMemoryStore
materialized useful?


>
> Despite the test failing, the PRFuelPersistency works as expected to
> save/restore a Pier kernel to/from a file. The file is 30 times smaller than
> the corresponding (uncompressed) SIXX file.
>

good! so it can be useful for scenarios where you don't need to be
cross-dialect or to have XML format.

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

Hi Eliot,
I am very happy to read your mail.

On Wed, Jun 15, 2011 at 3:29 PM, Eliot Miranda wrote:

> Hi Martin & Mariano,
>
> regarding filtering.  Yesterday my colleague Yaron and I successfully
> finished our port of Fuel to Newspeak and are successfully using it to save
> and restore our data sets; thank you, its a cool framework.  We had to
> implement two extensions, the first of which the ability to save and restore
> Newspeak classes, which is complex because these are instantiated classes
> inside instantiated Newspeak modules, not static Smalltalk classes in the
> Smalltalk dictionary.  The second extension is the ability to map specific
> objects to nil, to prune objects on the way out.  I want to discuss this
> latter extension.
>
> In our data set we have a set of references to objects that are logically
> not persistent and hence not to be saved.  I'm sure that this will be a
> common case.  The requirement is for the pickling system to prune certain
> objects, typically by arranging that when an object graph is pickled,
> references to the pruned objects are replaced by references to nil.  One way
> of doing this is as described below, by specifiying per-class lists of
> instance variables whose referents shoudl not be saved.  But this can be
> clumsy; there may be references to objects one wants to prune from e.g. more
> than one class, in which case one may have to provide multiple lists of the
> relevant inst vars; there may be references to objects one wants to prune
> from e.g. collections (e.g. sets and dictionaries) in which case the
> instance variable list approach just doesn't work.
>
> Here are two more general schemes.  VFirst, most directly, Fuel could
> provide two filters, implemented in the default mapper, or the core
> analyser.  One is a set of classes whose instances are not to be saved.  Any
> reference to an instance of a class in the toBePrunedClasses set is saved as
> nil.  The other is a set of instances that are not to be saved, and also any
> reference to an instance in the toBePruned set is saved as nil.  Why have
> both?  It can be convenient and efficient to filter by class (in our case we
> had many instances of a specific class, all of which should be filtered, and
> finding them could be time consuming), but filtering by class can be too
> inflexible, there may indeed be specific instances to exclude (thing for
> example of part of the object graph that functions as a cache; pruning the
> specific objects in the cache is the right thing to do; pruning all
> instances of classes whose instances exist in the cache may prune too much).
>
> As an example here's how we implemented pruning.  Our system is called
> Glue, and we start with a mapper for Glue objects, FLGlueMapper:
>
> FLMapper subclass: #FLGlueMapper
>  instanceVariableNames: 'prunedObjectClasses newspeakClassesCluster
> modelClasses'
> classVariableNames: ''
>  poolDictionaries: ''
> category: 'Fuel-Core-Mappers'
>
> It accepts newspeak objects and filters instances in the
> prunedObjectsClasses set, and as a side-effect collects certain classes that
> we need in a manifest:
>
> FLGlueMapper>>accepts: anObject
>  "Tells if the received object is handled by this analyzer.  We want to
> hand-off
>  instantiated Newspeak classes to the newspeakClassesCluster, and we want
>  to record other model classes.  We want to filter-out instances of any
> class
>  in prunedObjectClasses."
>  ^anObject isBehavior
> ifTrue:
> [(self isInstantiatedNewspeakClass: anObject)
>  ifTrue: [true]
> ifFalse:
> [(anObject inheritsFrom: GlueDataObject) ifTrue:
>  [modelClasses add: anObject].
> false]]
> ifFalse:
>  [prunedObjectClasses includes: anObject class]
>
> It prunes by mapping instances of the prunedObjectClasses to a special
> cluster.  It can do this in visitObject: since any newspeak objects it is
> accepting will be visited in its visitClassOrTrait: method (i.e. it's
> implicit that all arguments to visitObjects: are instances of the
> prunedObjectsClasses set).
>
> FLGlueMapper>>visitObject: anObject
>
> analyzer
> mapAndTrace: anObject
>  to: FLPrunedObjectsCluster instance
> into: analyzer clustersWithBaselevelObjects
>
> FLPrunedObjectsCluster is a specialization of the nil,true,false cluster
> that maps its objects to nil:
>
> FLNilTrueFalseCluster subclass: #FLPrunedObjectsCluster
>  instanceVariableNames: ''
> classVariableNames: ''
> poolDictionaries: ''
>  category: 'Fuel-Core-Clusters'
>
> FLPrunedObjectsCluster >>serialize: aPrunedObject on: aWriteStream
>
>  super serialize: nil on: aWriteStream
>
>
> So this would generalize by the analyser having an e.g. FLPruningMapper as
> the first mapper, and this having a prunedObjects and a priunedObjectClasses
> set and going something like this:
>
> FLPruningMapper>>accepts: anObject
> ^(prunedObjects includes: anObject) or: [prunedObjectClasses includes:
> anObject class]
>
> FLPruningMapper >>visitObject: anObject
> analyzer
> mapAndTrace: anObject
> 

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

I think the substitution should be done during the graph trace. Following
with the example, if a proxy replaces an object, the proxy represents a
subgraph that is appended and so it should be traced.

For that we should keep track of the substitutions. I'm not sure how complex
is that but is think it's not so difficult.

Seems to be a great idea, we have to try it. I like that avoids writing inst
var names as strings. I have no idea if with *slots* implemented then we
will be able to return inst vars as first-class objects... but anyway this
looks like the a nice solution.

So, we have this as a pending issue as well as the id virtualization. Thanks
for the ideas and the discussion!

Martin

On Fri, Jun 17, 2011 at 7:09 PM, Nicolas Cellier <
nicolas.cellier.aka.n...@gmail.com> wrote:

> 2011/6/17 Eliot Miranda :
> >
> >
> > On Fri, Jun 17, 2011 at 2:39 PM, Nicolas Cellier
> >  wrote:
> >>
> >> 2011/6/17 Eliot Miranda :
> >> >
> >> >
> >> > On Fri, Jun 17, 2011 at 1:26 AM, Martin Dias 
> >> > wrote:
> >> >>
> >> >> Hi Eliot,
> >> >> I am very happy to read your mail.
> >> >>
> >> >> On Wed, Jun 15, 2011 at 3:29 PM, Eliot Miranda
> >> >> 
> >> >> wrote:
> >> >>>
> >> >>> Hi Martin & Mariano,
> >> >>> regarding filtering.  Yesterday my colleague Yaron and I
> >> >>> successfully
> >> >>> finished our port of Fuel to Newspeak and are successfully using it
> to
> >> >>> save
> >> >>> and restore our data sets; thank you, its a cool framework.  We had
> to
> >> >>> implement two extensions, the first of which the ability to save and
> >> >>> restore
> >> >>> Newspeak classes, which is complex because these are instantiated
> >> >>> classes
> >> >>> inside instantiated Newspeak modules, not static Smalltalk classes
> in
> >> >>> the
> >> >>> Smalltalk dictionary.  The second extension is the ability to map
> >> >>> specific
> >> >>> objects to nil, to prune objects on the way out.  I want to discuss
> >> >>> this
> >> >>> latter extension.
> >> >>> In our data set we have a set of references to objects that are
> >> >>> logically
> >> >>> not persistent and hence not to be saved.  I'm sure that this will
> be
> >> >>> a
> >> >>> common case.  The requirement is for the pickling system to prune
> >> >>> certain
> >> >>> objects, typically by arranging that when an object graph is
> pickled,
> >> >>> references to the pruned objects are replaced by references to nil.
> >> >>>  One way
> >> >>> of doing this is as described below, by specifiying per-class lists
> of
> >> >>> instance variables whose referents shoudl not be saved.  But this
> can
> >> >>> be
> >> >>> clumsy; there may be references to objects one wants to prune from
> >> >>> e.g. more
> >> >>> than one class, in which case one may have to provide multiple lists
> >> >>> of the
> >> >>> relevant inst vars; there may be references to objects one wants to
> >> >>> prune
> >> >>> from e.g. collections (e.g. sets and dictionaries) in which case the
> >> >>> instance variable list approach just doesn't work.
> >> >>> Here are two more general schemes.  VFirst, most directly, Fuel
> could
> >> >>> provide two filters, implemented in the default mapper, or the core
> >> >>> analyser.  One is a set of classes whose instances are not to be
> >> >>> saved.  Any
> >> >>> reference to an instance of a class in the toBePrunedClasses set is
> >> >>> saved as
> >> >>> nil.  The other is a set of instances that are not to be saved, and
> >> >>> also any
> >> >>> reference to an instance in the toBePruned set is saved as nil.  Why
> >> >>> have
> >> >>> both?  It can be convenient and efficient to filter by class (in our
> >> >>> case we
> >> >>> had many instances of a specific class, all of which should be
> >> >>> filtered, and
> >> >>> finding them could be time consuming), bu

Re: [Pharo-project] Fuel - a fast object deployment tool

[Martin Dias]

I had some problems when I tried to write a package export tool with fuel. I
wanted to store the classes without method extensions of other packages.
Maybe with the objectToSerializeIn: idea I can write:

Class>>objectToSerializeIn: anFLAnalyser
^(anFLAnalyser shouldAvoidForeignProtocol: self)
ifFalse: [self]
ifTrue: [self copyWithoutForeignProtocol]

Cheers,
Martin

On Mon, Jun 20, 2011 at 1:48 AM, Martin Dias  wrote:

> I think the substitution should be done during the graph trace. Following
> with the example, if a proxy replaces an object, the proxy represents a
> subgraph that is appended and so it should be traced.
>
> For that we should keep track of the substitutions. I'm not sure how
> complex is that but is think it's not so difficult.
>
> Seems to be a great idea, we have to try it. I like that avoids writing
> inst var names as strings. I have no idea if with *slots* implemented then
> we will be able to return inst vars as first-class objects... but anyway
> this looks like the a nice solution.
>
> So, we have this as a pending issue as well as the id virtualization.
> Thanks for the ideas and the discussion!
>
> Martin
>
>
> On Fri, Jun 17, 2011 at 7:09 PM, Nicolas Cellier <
> nicolas.cellier.aka.n...@gmail.com> wrote:
>
>> 2011/6/17 Eliot Miranda :
>> >
>> >
>> > On Fri, Jun 17, 2011 at 2:39 PM, Nicolas Cellier
>> >  wrote:
>> >>
>> >> 2011/6/17 Eliot Miranda :
>> >> >
>> >> >
>> >> > On Fri, Jun 17, 2011 at 1:26 AM, Martin Dias 
>> >> > wrote:
>> >> >>
>> >> >> Hi Eliot,
>> >> >> I am very happy to read your mail.
>> >> >>
>> >> >> On Wed, Jun 15, 2011 at 3:29 PM, Eliot Miranda
>> >> >> 
>> >> >> wrote:
>> >> >>>
>> >> >>> Hi Martin & Mariano,
>> >> >>> regarding filtering.  Yesterday my colleague Yaron and I
>> >> >>> successfully
>> >> >>> finished our port of Fuel to Newspeak and are successfully using it
>> to
>> >> >>> save
>> >> >>> and restore our data sets; thank you, its a cool framework.  We had
>> to
>> >> >>> implement two extensions, the first of which the ability to save
>> and
>> >> >>> restore
>> >> >>> Newspeak classes, which is complex because these are instantiated
>> >> >>> classes
>> >> >>> inside instantiated Newspeak modules, not static Smalltalk classes
>> in
>> >> >>> the
>> >> >>> Smalltalk dictionary.  The second extension is the ability to map
>> >> >>> specific
>> >> >>> objects to nil, to prune objects on the way out.  I want to discuss
>> >> >>> this
>> >> >>> latter extension.
>> >> >>> In our data set we have a set of references to objects that are
>> >> >>> logically
>> >> >>> not persistent and hence not to be saved.  I'm sure that this will
>> be
>> >> >>> a
>> >> >>> common case.  The requirement is for the pickling system to prune
>> >> >>> certain
>> >> >>> objects, typically by arranging that when an object graph is
>> pickled,
>> >> >>> references to the pruned objects are replaced by references to nil.
>> >> >>>  One way
>> >> >>> of doing this is as described below, by specifiying per-class lists
>> of
>> >> >>> instance variables whose referents shoudl not be saved.  But this
>> can
>> >> >>> be
>> >> >>> clumsy; there may be references to objects one wants to prune from
>> >> >>> e.g. more
>> >> >>> than one class, in which case one may have to provide multiple
>> lists
>> >> >>> of the
>> >> >>> relevant inst vars; there may be references to objects one wants to
>> >> >>> prune
>> >> >>> from e.g. collections (e.g. sets and dictionaries) in which case
>> the
>> >> >>> instance variable list approach just doesn't work.
>> >> >>> Here are two more general schemes.  VFirst, most directly, Fuel
>> could
>> >> >>> provide two filters, implemented in the default mapper, or the co

[Pharo-project] [ANN] Fuel version 1.5 released

[Martin Dias]

I am happy to announce the new release version 1.5. Below is the list
of changes. In next days we will update the home page with current
state. For example, we have much more to show in benchmarks section.
Of course, we have many ideas and new issues to keep working on Fuel.

Again, I have to thank ESUG for sponsoring me for Fuel development
through the SummerTalk program.

Changes on 1.5:

- Initialize instances after materialization implementing
#fuelAfterMaterialization.
- Ignore transient instance variables overriding
#fuelIgnoredInstanceVariableNames.
- Limit serialization progress bar to update at most once every 250
milliseconds.
- Traits are serialized and materialized as a regular objects
(http://code.google.com/p/fuel/issues/detail?id=59).
- Removed mappers system cache (Eliot).
- Many new tests.
- Benchmarks: compare Fuel against StOMP, SRP, SIXX, Magma, EsAndEm
and Fuel with progress bar.
- Benchmarks: new samples, scripts, and CSV exporter.   
- Benchmarks: measure serialization stream size.
- Removed some redundant #fuelAccept:.
- Several aesthetic changes in protocols, categories and comments.
- Now #materialize answers the materialized root.
- Removed special cluster for HashedCollection.Now #rehash is sended
via #fuelAfterMaterialization.
- MethodContext serialization does not serialize temps
(http://code.google.com/p/fuel/issues/detail?id=61).
- Classes and Traits now serialize its environment
(testCreateWithSmalltalkGlobalsEnvironment).
- Fixed serialization of non-octet characters (testCharacter).
- Fixed a bug serializing the system dictionary.
FLWellKnownObjectsCluster handles it. (testSmalltalkGlobals).
- Fixed a bug serializing class variables (testClassVariable).
- Fixed a bug when serializing a collection with size greater than
1^16 with repeated elements inside.
- Fixed testTwoCompiledMethodsReferencingSameClassVariable.
- Optimization: instanceIndexes are directly created with the exact
size (Henrik).

Re: [Pharo-project] [Esug-list] [ANN] Fuel version 1.5 released

[Martin Dias]

Hi Eliot,
Yes, it would be better to include more information. I will put the
home page url too.

Thanks
Martin

On Wed, Jul 6, 2011 at 1:16 PM, Eliot Miranda  wrote:
> Hi Martin,
>     I wonder could you repost your announcement but this time set the
> example of including a brief description of what Fuel is for those who don't
> know.  So the announcement reads something like:
> I am happy to announce the new release version 1.5. Fuel is a binary
> serialization system for Smalltalk that provides extremely fast
> deserialization.
> Below is the list of changes. In next days we will update the home page with
> current
> state. For example, we have much more to show in benchmarks section.
> Of course, we have many ideas and new issues to keep working on Fuel.
> ...
>
> On Wed, Jul 6, 2011 at 1:27 AM, Martin Dias  wrote:
>>
>> I am happy to announce the new release version 1.5. Below is the list
>> of changes. In next days we will update the home page with current
>> state. For example, we have much more to show in benchmarks section.
>> Of course, we have many ideas and new issues to keep working on Fuel.
>>
>> Again, I have to thank ESUG for sponsoring me for Fuel development
>> through the SummerTalk program.
>>
>> Changes on 1.5:
>>
>> - Initialize instances after materialization implementing
>> #fuelAfterMaterialization.
>> - Ignore transient instance variables overriding
>> #fuelIgnoredInstanceVariableNames.
>> - Limit serialization progress bar to update at most once every 250
>> milliseconds.
>> - Traits are serialized and materialized as a regular objects
>> (http://code.google.com/p/fuel/issues/detail?id=59).
>> - Removed mappers system cache (Eliot).
>> - Many new tests.
>> - Benchmarks: compare Fuel against StOMP, SRP, SIXX, Magma, EsAndEm
>> and Fuel with progress bar.
>> - Benchmarks: new samples, scripts, and CSV exporter.
>> - Benchmarks: measure serialization stream size.
>> - Removed some redundant #fuelAccept:.
>> - Several aesthetic changes in protocols, categories and comments.
>> - Now #materialize answers the materialized root.
>> - Removed special cluster for HashedCollection.Now #rehash is sended
>> via #fuelAfterMaterialization.
>> - MethodContext serialization does not serialize temps
>> (http://code.google.com/p/fuel/issues/detail?id=61).
>> - Classes and Traits now serialize its environment
>> (testCreateWithSmalltalkGlobalsEnvironment).
>> - Fixed serialization of non-octet characters (testCharacter).
>> - Fixed a bug serializing the system dictionary.
>> FLWellKnownObjectsCluster handles it. (testSmalltalkGlobals).
>> - Fixed a bug serializing class variables (testClassVariable).
>> - Fixed a bug when serializing a collection with size greater than
>> 1^16 with repeated elements inside.
>> - Fixed testTwoCompiledMethodsReferencingSameClassVariable.
>> - Optimization: instanceIndexes are directly created with the exact
>> size (Henrik).
>>
>> ___
>> Esug-list mailing list
>> esug-l...@lists.esug.org
>> http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org
>
>
>
> --
> best,
> Eliot
>

[Pharo-project] [ANN] Fuel version 1.5 released (bis)

[Martin Dias]

Hi. I am happy to announce the new release version 1.5. At the end of this
message is the list of changes.

Fuel is a binary serialization system for Smalltalk that provides extremely
fast deserialization. More information on our home
page
. In next days we will update it. We have much more to show especially in
benchmarks section. We have many ideas and new
issuesto keep working on
Fuel.

You can download this version in Pharo with:

Gofer new
squeaksource: 'Fuel';
package: 'ConfigurationOfFuel';
load.
((Smalltalk at: #ConfigurationOfFuel) project version: '1.5') load.

Again, I have to thank ESUG for sponsoring me for Fuel development through
the SummerTalk program.

Changes:

- Initialize instances after materialization implementing
#fuelAfterMaterialization.
- Ignore transient instance variables overriding
#fuelIgnoredInstanceVariableNames.
- Limit serialization progress bar to update at most once every 250
milliseconds.
- Traits are serialized and materialized as a regular objects (
http://code.google.com/p/fuel/issues/detail?id=59).
- Removed mappers system cache (Eliot).
- Many new tests.
- Benchmarks: compare Fuel against StOMP, SRP, SIXX, Magma, EsAndEm and Fuel
with progress bar.
- Benchmarks: new samples, scripts, and CSV exporter.
- Benchmarks: measure serialization stream size.
- Removed some redundant #fuelAccept:.
- Several aesthetic changes in protocols, categories and comments.
- Now #materialize answers the materialized root.
- Removed special cluster for HashedCollection.Now #rehash is sended via
#fuelAfterMaterialization.
- MethodContext serialization does not serialize temps (
http://code.google.com/p/fuel/issues/detail?id=61).
- Classes and Traits now serialize its environment
(testCreateWithSmalltalkGlobalsEnvironment).
- Fixed serialization of non-octet characters (testCharacter).
- Fixed a bug serializing the system dictionary. FLWellKnownObjectsCluster
handles it. (testSmalltalkGlobals).
- Fixed a bug serializing class variables (testClassVariable).
- Fixed a bug when serializing a collection with size greater than 1^16 with
repeated elements inside.
- Fixed testTwoCompiledMethodsReferencingSameClassVariable.
- Optimization: instanceIndexes are directly created with the exact size
(Henrik).

[Pharo-project] [ANN] Fuel version 1.5 released (bis)

[Martin Dias]

Hi. I am happy to announce the new release version 1.5. At the end of this
message is the list of changes.

Fuel is a binary serialization system for Smalltalk that provides extremely
fast deserialization. More information on our home
page
. In next days we will update it. We have much more to show especially in
benchmarks section. We have many ideas and new
issuesto keep working on
Fuel.

You can download this version in Pharo with:

Gofer new
squeaksource: 'Fuel';
package: 'ConfigurationOfFuel';
load.
((Smalltalk at: #ConfigurationOfFuel) project version: '1.5') load.

Again, I have to thank ESUG for sponsoring me for Fuel development through
the SummerTalk program.

Changes:

- Initialize instances after materialization implementing
#fuelAfterMaterialization.
- Ignore transient instance variables overriding
#fuelIgnoredInstanceVariableNames.
- Limit serialization progress bar to update at most once every 250
milliseconds.
- Traits are serialized and materialized as a regular objects (
http://code.google.com/p/fuel/issues/detail?id=59).
- Removed mappers system cache (Eliot).
- Many new tests.
- Benchmarks: compare Fuel against StOMP, SRP, SIXX, Magma, EsAndEm and Fuel
with progress bar.
- Benchmarks: new samples, scripts, and CSV exporter.
- Benchmarks: measure serialization stream size.
- Removed some redundant #fuelAccept:.
- Several aesthetic changes in protocols, categories and comments.
- Now #materialize answers the materialized root.
- Removed special cluster for HashedCollection.Now #rehash is sended via
#fuelAfterMaterialization.
- MethodContext serialization does not serialize temps (
http://code.google.com/p/fuel/issues/detail?id=61).
- Classes and Traits now serialize its environment
(testCreateWithSmalltalkGlobalsEnvironment).
- Fixed serialization of non-octet characters (testCharacter).
- Fixed a bug serializing the system dictionary. FLWellKnownObjectsCluster
handles it. (testSmalltalkGlobals).
- Fixed a bug serializing class variables (testClassVariable).
- Fixed a bug when serializing a collection with size greater than 1^16 with
repeated elements inside.
- Fixed testTwoCompiledMethodsReferencingSameClassVariable.
- Optimization: instanceIndexes are directly created with the exact size
(Henrik).

[Pharo-project] FloatArray inaccuracy

[Martin Dias]

Hi

I found that FloatArray>>at:put: is inaccurate:

| floats |
floats := FloatArray new: 1.
floats at: 1 put: 11.22321.
floats at: 1 " > 11.223210334777832 "


Do you know this issue?


This is the implementation of at:put::

at: index put: value

value isFloat
ifTrue:[self basicAt: index put: value asIEEE32BitWord]
ifFalse:[self at: index put: value asFloat].
^value

Thanks in advance
Martin

Re: [Pharo-project] FloatArray inaccuracy

[Martin Dias]

If forgot to specify that I run this on Pharo 1.3 build #226 in Windows with
:

Smalltalk vmVersion > 'Croquet Closure Cog VM [CoInterpreter
VMMaker.oscog-eem.56]'

Martin

On Thu, Jul 28, 2011 at 3:30 AM, Martin Dias  wrote:

> Hi
>
> I found that FloatArray>>at:put: is inaccurate:
>
> | floats |
> floats := FloatArray new: 1.
> floats at: 1 put: 11.22321.
> floats at: 1 " > 11.223210334777832 "
>
>
> Do you know this issue?
>
>
> This is the implementation of at:put::
>
> at: index put: value
> 
> value isFloat
> ifTrue:[self basicAt: index put: value asIEEE32BitWord]
> ifFalse:[self at: index put: value asFloat].
> ^value
>
> Thanks in advance
> Martin
>

Re: [Pharo-project] Does anyone has FUEL running in Pharo 1.3?

[Martin Dias]

Hi Friedrich

It should run. How can I reproduce your error?

Martin

On Thu, Jul 28, 2011 at 3:38 AM, Friedrich Dominicus <
fr...@q-software-solutions.de> wrote:

> Just wondering, because while trying to load it I get an error message
>
> Regards
> Friedrich
>
> --
> Q-Software Solutions GmbH; Sitz: Bruchsal; Registergericht: Mannheim
> Registriernummer: HRB232138; Geschaeftsfuehrer: Friedrich Dominicus
>
>

Re: [Pharo-project] Does anyone has FUEL running in Pharo 1.3?

[Martin Dias]

On Thu, Jul 28, 2011 at 4:15 AM, Friedrich Dominicus <
fr...@q-software-solutions.de> wrote:

> Martin Dias  writes:
>
> > Hi Friedrich
> >
> > It should run. How can I reproduce your error?
> Well I probably had not a plain Pharo 1.3 (was not aware of that)
> but I reinstalled a fresh Pharo 1.3 and it simple loaded.
>
> I'm sorry for the noise
>
>
Fine. You are welcome.



>  Friedrich
>
> --
> Q-Software Solutions GmbH; Sitz: Bruchsal; Registergericht: Mannheim
> Registriernummer: HRB232138; Geschaeftsfuehrer: Friedrich Dominicus
>
>

Re: [Pharo-project] FloatArray inaccuracy

[Martin Dias]

On Thu, Jul 28, 2011 at 4:54 AM, Igor Stasenko  wrote:

> Float numbers are inexact and always been.
>
> Maybe you meant that they are more inexact than you expected? :)
>
> FloatArray stores floats using 32-bit floating point precision format.
> While Float(s) using 64 bits.
> No surprise that values stored in FloatArray are less precise.
>
>
Ahhh! you are right... I was storing 64 bits:

serialize: aFloat on: aWriteStream
aWriteStream
nextNumber: 4 put: (aFloat at: 1);
nextNumber: 4 put: (aFloat at: 2).

materializeFrom: aReadStream
^ (Float new: 2)
at: 1 put: (aReadStream nextNumber: 4);
at: 2 put: (aReadStream nextNumber: 4);
yourself.

that's why I see a difference!
thanks!!!


On 28 July 2011 08:34, Martin Dias  wrote:
> > If forgot to specify that I run this on Pharo 1.3 build #226 in Windows
> with
> > :
> >
> > Smalltalk vmVersion > 'Croquet Closure Cog VM [CoInterpreter
> > VMMaker.oscog-eem.56]'
> >
> > Martin
> >
> > On Thu, Jul 28, 2011 at 3:30 AM, Martin Dias 
> wrote:
> >>
> >> Hi
> >>
> >> I found that FloatArray>>at:put: is inaccurate:
> >>
> >> | floats |
> >> floats := FloatArray new: 1.
> >> floats at: 1 put: 11.22321.
> >> floats at: 1 " > 11.223210334777832 "
> >>
> >>
> >> Do you know this issue?
> >>
> >>
> >> This is the implementation of at:put::
> >>
> >> at: index put: value
> >> 
> >> value isFloat
> >> ifTrue:[self basicAt: index put: value asIEEE32BitWord]
> >> ifFalse:[self at: index put: value asFloat].
> >> ^value
> >>
> >> Thanks in advance
> >> Martin
> >
> >
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

[Pharo-project] [ANN] Fuel release version 1.6

[Martin Dias]

Hello!

I want to announce the new release version number 1.6 of Fuel, a
binary serialization framework for Pharo that provides extremely fast
deserialization. More information on our home
page. Again,
I thank ESUG for sponsoring me through the
SummerTalkprogram.

You can download this version in Pharo 1.2.1 with:

Gofer new
squeaksource: 'Fuel';
package: 'ConfigurationOfFuel';
load.
((Smalltalk at: #ConfigurationOfFuel) project version: '1.6') load.

This version does not add any new concrete functionality but has two main
achievements: a performance improvement and a design clean up.

Many changes included in this release are part of a design clean up. They
are a necessary step forward to new functionalities that are coming soon.
They make code more understandable. They enable simpler extension. Happily
they also helped to improve serialization performance by reducing
unnecessary interactions. And it is specially important a bottleneck in
graph analysis stage we have successfully attacked, obtaining great results.

Details:
- Removed explicit references to cluster classes scattered around the
system. For example, in previous versions Float>>fuelCluster has an explicit
reference to FLFloatCluster. This version replaces this by a double dispatch
interaction.
- Clusters: Better reification. Removed annoying IDs. Removed Singleton
instance creation mechanism, now mappers manage uniqueness in a better way.
- Mappers: They were simplified. New mapping machanism inspired on Chain of
Responsibility pattern. The global objects detection has been optimized
considerably.
- Analyzer: Part of its behavior and state has been moved to a new
abstraction called 'clusterization'.

Best regards,
Martín Dias

Re: [Pharo-project] Fuel or image segments (project files)

[Martin Dias]

Hi Dimitri,

I think image segments are supported in same way in Pharo but I'm not sure.

About Fuel, it is in active development. It currently works and has several
release versions but it doesn't replace completely
ImageSegment. Hopefully we might be capable of doing that in some months...
at the moment you can find some limitations, especially if you were using
DiskProxy (do you?).

There is more info, including how to install and try it, here:
http://rmod.lille.inria.fr/web/pier/software/Fuel

Cheers,
Martín


On Wed, Sep 14, 2011 at 6:59 PM, Dimitry Golubovsky wrote:

> Hi,
>
> The application I am working on (which currently runs under Squeak VM
> on Android) will be migrated to Pharo. The current version heavily
> relies upon use of image segments (projects) which provides a simple
> (though slow) way to save/restore the application data not touching
> the image (the Squeak port is unable to save images anyway, Cog can do
> this, but I'd rater not touch the image) with versioning.
>
> Are image segments supported in Pharo Core same way? I saw a message
> about stopping the support of image segments in future versions of
> Pharo in favor of Fuel. What is current status of Fuel development? Is
> it usable?
>
> Thanks.
>
> --
> Dimitry Golubovsky
>
> Anywhere on the Web
>
>

Re: [Pharo-project] Should a serializer do something in particular with weak references?

[Martin Dias]

Hi

> if you look at the problem in wider perspective, then right question
> to ask, is how to deal
> with objects which are register themselves in global facilities, like
> weak finalization,
> startup/shutdown lists etc.

Fuel can handle globals... but I don't understand how can weak objects be
accessed as global. Could you give a snippet exemplifying, please?

Martin

On Tue, Sep 20, 2011 at 11:03 PM, Igor Stasenko  wrote:

> On 21 September 2011 03:15, Schwab,Wilhelm K 
> wrote:
> > But how does Fuel "know" that the objects are not otherwise referenced?
>  The idea of ignoring weak collections sounds a lot like a compiler
> optimization that sounds efficient and harmless - sometimes they cause huge
> problems.  Think of it this way, if the objects are not otherwise
> referenced, sure they will be gc's soon - why are are they still around???
>
> the main problem i think is to make fuel to know which objects are
> belong to your model, and which is not.
> because in Levente's suggestion, this makes a difference between
> either serializing a pointer or just replacing it with nil.
>
> But then, once you introduce "this object belongs to my model" logic,
> a new question immediately arises: what if some object strongly
> referencing an object outside your model?
> So, the problem is more general than just do it selectively only for
> weak containers. Either you doing it uniformly (providing facilities
> to deal with everything which outside your domain/model somehow),
> or you just walk the graph and storing everything (treating all object
> found as ones , who belong to your model). And there is nothing
> special to do with weak containers.
>
> > The comments re globally registered objects are well taken too.
>  Dolphin's proxies are a great way to handle such things: some objects
> delegate their serialization to a proxy that contains logic to do global
> registration, etc.
> >
> > Bill
> >
> >
> > 
> > From: pharo-project-boun...@lists.gforge.inria.fr [
> pharo-project-boun...@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier
> [nicolas.cellier.aka.n...@gmail.com]
> > Sent: Tuesday, September 20, 2011 6:08 PM
> > To: Pharo-project@lists.gforge.inria.fr
> > Subject: Re: [Pharo-project] Should a serializer do something in
> particular with weak references?
> >
> > 2011/9/21 Levente Uzonyi :
> >> On Tue, 20 Sep 2011, Mariano Martinez Peck wrote:
> >>
> >>> Hi guys. Martin and I were discussing whether we should do something in
> >>> particular in Fuel with Weak references, such as WeakArray. Should we
> >>> serialize it the same way we serialize all objects, or should we do
> >>> something special?  what could be the impact of materializing such weak
> >>> references in a new image for example?
> >>
> >> You should ignore (serialize nil instead of them) the elements of a weak
> >> object (#weakSubclass:...) which aren't referenced strongly from other
> >> serialized objects.
> >>
> >>
> >> Levente
> >>
> >
> > Ach, you commited shorter and quicker ;)
> >
> > Nicolas
> >
> >>>
> >>> Thanks your sharing your thoughts.
> >>>
> >>> --
> >>> Mariano
> >>> http://marianopeck.wordpress.com
> >>>
> >>
> >>
> >
> >
> >
>
>
>
> --
> Best regards,
> Igor Stasenko.
>
>

Re: [Pharo-project] [squeak-dev] Any serializer can customize serialization strategy?

[Martin Dias]

Hi Hernan,

Could your "specific Strings" be instances of a variant of RemoteString that
compresses and decompresses? Then they will occupy less memory in your image
and it would be easier to serializer and deserialize the proxy object.

Thanks for the feedback, I also don't like the visitor interface we have for
customizations. In fact, I think it is not a visitor but a double-dispatch.
And I would prefer to merge the two hooks Mariano proposed to something
like:

String >>
fuelMapOn: aDefaultMapper (ExternalCompressor shouldCompressString: self)
ifTrue: [aDefaultMapper mapSubstitution: self by: (ExternalCompressor
compress: self)] ifFalse: [super fuelMapOn: aDefaultMapper]


Do you like more this way?

Martin


On Tue, Oct 11, 2011 at 4:10 PM, Mariano Martinez Peck <
marianop...@gmail.com> wrote:

>
>
> On Tue, Oct 11, 2011 at 12:56 AM, Hernán Morales Durand <
> hernan.mora...@gmail.com> wrote:
>
>> Thanks Mariano,
>>
>> That's pretty much what I want, but... why to expose the visitor
>> interface? ;)
>>
>
> Good question. Honesty, it was the only way we have found to do it with
> Fuel. I think (but I am not sure) that one of the drawbacks of the pickle
> format is that it analysis phase (as we call it in Fuel) make some hooks
> more complicated to implement than a classic serializer.
>
>
>> Just to give a little feedback of what I have in my mind:
>>
>> mySerializer setEngine: ExternalCompressor for: ( MyClass -> #myStringIVar
>> )
>>
>>
> Well, you can also create an specific Fuel cluster for MyClass that does
> exactly that. Basically you have to create FLMyClassCluster, implement
> methods like #serializeWith: and #materializeFrom:  and there you
> serialize/materialize the specific #myStringIVar with a special way and the
> rest as normal. Then you subclass the FLDefaultMapper, say YourMapper and
> you implement #visitXXX
> Then you have to implement MyClass >> visitXXX.
> Of course, you should be able to generalize such cluster and be able to
> reuse it for different classes or instVars.
>
> I am not saying this is the best way. I am just telling you one possible
> solution you can do by extending Fuel for your needs. Of course, you may
> need to understand how Fuel works first. But of course, we can help if you
> want.
>
>
>> or
>>
>> mySerializer setEngine: ExternalCompressor for: incomingStream
>>
>> Now with machines generating daily 6 TBytes of raw data, one of the
>> keys will be data compression.
>>
>
>
> it looks like ;)
>
>
>
>> 2011/10/10 Mariano Martinez Peck :
>> >
>> >
>> > On Mon, Oct 10, 2011 at 9:35 PM, Hernán Morales Durand
>> >  wrote:
>> >>
>> >> Is there any serializer (Fuel/StOMP/Ma object
>> >> serialization/BOSS/SRP/SIXX??) which let me attach an external
>> >> compressor/decompressor for specific Strings to the
>> >> serialization/deserialization process?
>> >
>> > In Fuel you can do:
>> >
>> > String >> fuelSubstitution
>> >   ^ ExternalCompressor compress: self
>> >
>> > String >> fuelAccept: aVisitor
>> >^ (ExternalCompressor shouldCompressString: self)
>> >ifTrue: [aVisitor visitSubstitution: self]
>> >ifFalse: [super fuelAccept: aVisitor]
>> >
>> > But the problem is that if #compress: answers an instance of String, you
>> > should be careful that if #shouldCompressString:  asnwers true to the
>> > compressed string, then you will end up in an infinitive loop.
>> >
>> > See: http://rmod.lille.inria.fr/web/pier/software/Fuel/Hooks and
>> > FLHookedSubstitutionTest
>> >
>> > I think StOMP should support this as well:
>> > http://stomp.smalltalk-users.jp/home/how-to-use-stomp/hook-methods
>> >
>> > The previous hook you let you replace the original String with its
>> > compression during serialization. Which make sense because if you are
>> going
>> > to compress it is better to do it during serialization.
>> >
>> >> With external I mean it could
>> >> be in a separate binary (I could arrange the binding though).
>> >
>> > I am not sure if I understood.
>> >
>> >
>> >
>> >>
>> >> Thanks
>> >>
>> >> Hernán
>> >>
>> >
>> >
>> >
>> > --
>> > Mariano
>> > http://marianopeck.wordpress.com
>> >
>> >
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

[Pharo-project] [ANN] Fuel release version 1.7

[Martin Dias]

Hi folks!

We have released the new version 1.7 of Fuel, a binary serialization
framework for Pharo that provides extremely fast deserialization. More
information on our home
page. Again,
I thank ESUG for sponsoring me through the
SummerTalk
 program.

You can download this version in Pharo 1.2.1 or 1.3 with:

Gofer new
squeaksource: 'Fuel';
package: 'ConfigurationOfFuel';
load.
((Smalltalk at: #ConfigurationOfFuel) project version: '1.7') load.


The list of changes includes performance optimizations, design clean-ups,
and new features.

- The FLSerializer and FLMaterializer API has changed. They are no longer
implementing the algorithms but are a kind of Façade.
- Developer guides with examples: Getting
Startedand
Customizing
the graph
.
- Serialization substitutions: "Store this object instead of me."
- Global sends: "Restore me by sending this selector to this global"
- Versioning the stream: We prefix the stream with a version number that
should match when loading.
- Performance optimizations on instances of:
-- Word-like classes. (We thank a lot to Henrik Sperre Johansen for your
help!)
-- ByteString and Symbol.
-- Date, Time, Duration and DateAndTime.
-- Point and Rectangle.
-- MethodDictionary. Now materialization is 2000x faster, thanks to its new
rehash without become.
- Huge clean-up in Tests package.

Cheers!
Martín Dias

Re: [Pharo-project] [squeak-dev] Any serializer can customize serialization strategy?

[Martin Dias]

On Fri, Oct 14, 2011 at 2:40 AM, Hernán Morales Durand <
hernan.mora...@gmail.com> wrote:

> Hi Martin
>
> 2011/10/13 Martin Dias :
> > Hi Hernan,
> > Could your "specific Strings" be instances of a variant of RemoteString
> that
> > compresses and decompresses? Then they will occupy less memory in your
> image
> > and it would be easier to serializer and deserialize the proxy object.
>
> I'd like to keep my objects cross-dialect, so if that's not an issue
> it would be ok to use a RemoteString. I don't see why it would be
> easier to serialize/deserialize?
>

I don't know it has sense, also it depends on your case... but if
MyRemoteString just has a instVar 'filename' or 'fileIndex' to fetch the
real string from disk when necessary... it should be easy to serialize
instances of such MyRemoteString.


>
> > Thanks for the feedback, I also don't like the visitor interface we have
> for
> > customizations. In fact, I think it is not a visitor but a
> double-dispatch.
> > And I would prefer to merge the two hooks Mariano proposed to something
> > like:
> > String >>
> > fuelMapOn: aDefaultMapper (ExternalCompressor shouldCompressString: self)
> > ifTrue: [aDefaultMapper mapSubstitution: self by: (ExternalCompressor
> > compress: self)] ifFalse: [super fuelMapOn: aDefaultMapper]
> >
> > Do you like more this way?
>
> Absolutely :)
>

thanks... probably for Fuel 1.8!


>
> > Martin
> >
> > On Tue, Oct 11, 2011 at 4:10 PM, Mariano Martinez Peck
> >  wrote:
> >>
> >>
> >> On Tue, Oct 11, 2011 at 12:56 AM, Hernán Morales Durand
> >>  wrote:
> >>>
> >>> Thanks Mariano,
> >>>
> >>> That's pretty much what I want, but... why to expose the visitor
> >>> interface? ;)
> >>
> >> Good question. Honesty, it was the only way we have found to do it with
> >> Fuel. I think (but I am not sure) that one of the drawbacks of the
> pickle
> >> format is that it analysis phase (as we call it in Fuel) make some hooks
> >> more complicated to implement than a classic serializer.
> >>
> >>>
> >>> Just to give a little feedback of what I have in my mind:
> >>>
> >>> mySerializer setEngine: ExternalCompressor for: ( MyClass ->
> >>> #myStringIVar )
> >>>
> >>
> >> Well, you can also create an specific Fuel cluster for MyClass that does
> >> exactly that. Basically you have to create FLMyClassCluster, implement
> >> methods like #serializeWith: and #materializeFrom:  and there you
> >> serialize/materialize the specific #myStringIVar with a special way and
> the
> >> rest as normal. Then you subclass the FLDefaultMapper, say YourMapper
> and
> >> you implement #visitXXX
> >> Then you have to implement MyClass >> visitXXX.
> >> Of course, you should be able to generalize such cluster and be able to
> >> reuse it for different classes or instVars.
> >>
> >> I am not saying this is the best way. I am just telling you one possible
> >> solution you can do by extending Fuel for your needs. Of course, you may
> >> need to understand how Fuel works first. But of course, we can help if
> you
> >> want.
> >>
> >>>
> >>> or
> >>>
> >>> mySerializer setEngine: ExternalCompressor for: incomingStream
> >>>
> >>> Now with machines generating daily 6 TBytes of raw data, one of the
> >>> keys will be data compression.
> >>>
> >>
> >> it looks like ;)
> >>
> >>
> >>>
> >>> 2011/10/10 Mariano Martinez Peck :
> >>> >
> >>> >
> >>> > On Mon, Oct 10, 2011 at 9:35 PM, Hernán Morales Durand
> >>> >  wrote:
> >>> >>
> >>> >> Is there any serializer (Fuel/StOMP/Ma object
> >>> >> serialization/BOSS/SRP/SIXX??) which let me attach an external
> >>> >> compressor/decompressor for specific Strings to the
> >>> >> serialization/deserialization process?
> >>> >
> >>> > In Fuel you can do:
> >>> >
> >>> > String >> fuelSubstitution
> >>> >   ^ ExternalCompressor compress: self
> >>> >
> >>> > String >> fuelAccept: aVisitor
> >>> >^ (ExternalCompressor shouldCompressString: self)
> >>> 

Re: [Pharo-project] When will Fuel file format stabilize?

[Martin Dias]

I like the ideas of Stef and Ben of loading multiple fuel versions at the
same time. Do you know something done in that sense?

Martín

On Sat, Oct 15, 2011 at 7:09 AM, Mariano Martinez Peck <
marianop...@gmail.com> wrote:

>
>
> On Sat, Oct 15, 2011 at 11:49 AM, Philippe Marschall wrote:
>
>> On 11.10.2011 21:51, Mariano Martinez Peck wrote:
>> >
>> >
>> > On Tue, Oct 11, 2011 at 8:55 AM, Philippe Marschall
>> > > > > wrote:
>> >
>> > On 10/08/2011 10:42 PM, Mariano Martinez Peck wrote:
>> > > s
>> > >
>> > >>>
>> > >>> This is IMHO more than necessary for Fuel to become a production
>> > ready
>> > >>> serializer and I'd say Fuel is now "old enough" to become such
>> :)
>> > >>
>> > >> Yes.
>> > >> Now what I would love is that even if fuel changes that the
>> > evolution of
>> > >> information
>> > >> is taken into account because like that it will be exercised for
>> > real.
>> > >>
>> > >>
>> > > No, that's impossible, and if posible, it is not worth it.
>> > Migrating from an
>> > > old format to a new one is extremelly complicated and innecessary.
>> The
>> > > easiest way to solve this is to take the correct version of Fuel,
>> > > materialize the graph from the stream, load new version of Fuel,
>> and
>> > > seriaize it again. That the easiest, more secure, and more
>> practical
>> > > approach I can see.
>> >
>> > That is horribly naïve an excludes fuel from a lot of use cases. You
>> > can't use fuel for "archiving" objects outside of the image because
>> you
>> > will never know whether you will be able to read them in again
>> because
>> > the format changes. You will always need to have "live" ones in the
>> > image.
>> >
>> >
>> > No. That's incorrect. You won't be able to do that ONLY if you update
>> > Fuel to a new image that breaks format.
>> > You can still continue to use the same version and you will never have
>> > that problem. So, again, why you need to update Fuel?
>>
>> Because it's old software. Bugs may not get fixed. It may not work in
>> newer Pharo versions. I may have dependencies on other libraries that
>> may require a new version of Fuel. You name it.
>>
>>
>
> I understand. What I mean is that depending on the changes and the amount
> of work, you can just adapt Fuel to new versions of Pharo but without
> changing its format.
> I meansay you were in Fuel 1.4. You don't need to move to Fuel 1.8 just
> because. You can just try to fix 1.4 to make it work in latest pharo.
> That's EXACTLY what we do with ReferenceStream and friends. The only
> difference is that the Pharo team does that because it is in the core.
> But yes, it may happen that ideed you will require a new version of Fuel.
>
>
>
>> > Why SmartRefStream does not have this problem?  because it hasn't
>> > changed in the last 10 years.
>> > So..do the same, take an specific Fuel version and keep it for 10 years.
>> > Just update it to make it work in Pharo without changing the format and
>> > you are done.
>> >
>> >
>> >
>> > That means you can't use fuel for anything Monticello related
>> because
>> > you may never be able to load those versions in again because the
>> file
>> > format has changed in the mean time.
>> >
>> >
>> > I guess that in the end, if someone can really do something for
>> > Monticello on top of Fuel it will be like 2 years from now, and at some
>> > point Fuel format will be stable.
>> > And as Stef says...you always have the code there in case of problems.
>>
>> Monticello is just an example for a case where I want to store objects
>> outside of the image.
>>
>>
> Well, then you can just wait until we finish. Give us one year more.
>  Instead of doing 1-year-long releases, we like to do 3 months releases.
> But again, that doesn't mean you have to update...
>
>
>
>
>> Cheers
>> Philippe
>>
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

Re: [Pharo-project] When will Fuel file format stabilize?

[Martin Dias]

Hi

I followed with interest the discussion but without many interventions. I
want to explain an idea I have for supporting backward-compatibility without
neither stabilizing the format nor getting the current code dirty. Maybe
it's a bit crazy but I think it can be a practical solution. I would like to
know your opinions.

We can offer:

- Maintain a *compatibility package* for each released version, that can
co-exist in the same image. For example Fuel1_5, Fuel1_6, etc. Obviously,
the current version always has the nice clean name (Fuel).
- Provide a facade that works like "Fuel load: aFile asOfVersion: '1.4'" who
interacts with the compatibility packages.


To generate each *compatibility package*, we rename several elements of the
original Fuel released package, this way:
  * Extension methods: Append at beggining of each extension method selector
a prefix. For example: #fuelAccept: --> #fuel1_6_fuelAccept:
  * Classes: Append to each one some version identifier. For example:
FLSerializer --> FLSerializer_1_6
  * Methods: Fix each reference to a class or extension method selector THAT
BELONGS to the package. For example: "FLSerializer new" -->
"FLSerializer_1_6 new".


I think it could be not that hard to make such renames automatically with a
small tool. Then, we can simultaneously maintain each *compatibility
package* (Fuel1_5, Fuel1_6, etc.) when Pharo evolves:

Our Jenkins will run tests for all the compatibility Fuel packages. For
example... Does Fuel1_7 stop working in Pharo1.5? then (if it's possible) we
fix it, save the monticello version and mark it in our ConfigurationOfFuel as
"this is the stable monticello version of fuel 1.7 in pharo 1.5".


Now that I see how many lines I took to explain it, I think it's a bit insane.
I hope to read some opinions!

Bests,
Martín

On Thu, Oct 20, 2011 at 6:00 PM, Schwab,Wilhelm K wrote:

> Mariano,
>
> I gave you a pretty tough time :)   It appears that you understand that I
> did so with the best of intentions.  Thanks for listening.
>
> One problem that I do *not* think you need to solve is that of old software
> talking to new software.  Someone wanting forward compatibility is probably
> going to use XML or something similar to get a timeless protocol that will
> be of necessity very simple.  A good serializer is hard to write, and
> forward-compatibility is probably too much to ask, particularly when
> efficiency is a stated goal.
>
> A backwardly-compatible Fuel will be most welcome and useful.
>
> Bill
>
>
> 
> From: pharo-project-boun...@lists.gforge.inria.fr [
> pharo-project-boun...@lists.gforge.inria.fr] On Behalf Of Mariano Martinez
> Peck [marianop...@gmail.com]
> Sent: Thursday, October 20, 2011 7:00 AM
> To: Pharo-project@lists.gforge.inria.fr
> Subject: Re: [Pharo-project] When will Fuel file format stabilize?
>
> On Thu, Oct 20, 2011 at 12:54 PM, Ben Coman  b...@openinworld.com>> wrote:
> Mariano Martinez Peck wrote:
> Well...thanks guys. The point is taken. Thanks Philip, Bill and all who
> gave feedback and suggestions. We are aware of this and we are already
> discussing with Martin possible solutions and analyzing all the things that
> has been said in this thread.
> We will keep you informed.
>
> Cheers
> Just one thing I hadn't seen discussed.  Is Fuel possibly a way to exchange
> objects/data between users/systems ?
>
> Yes
>
> In which case it would have to deal with different systems having different
> versions of Fuel.
>
>
> Exactly. It doesn't matter whether the different versions of Fuel happens
> in your own image or in different systems. So yes, it is the same problem.
> Note this is not only a problem for Fuel, but for any serializer which does
> not manages full backward compatibility between versions.
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>

Re: [Pharo-project] how should Pier file structures be serialized with Fuel?

[Martin Dias]

Hello!

On Thu, Nov 17, 2011 at 1:56 AM, Yanni Chiu  wrote:

> An earlier version of Fuel did not having the substitution feature. So, I
> had custom mapping code to pre-process the structures before passing to
> Fuel for serialization, and to post-process the deserialized structures.
> The main purpose of the code was to nil out the PUSecurity objects, because
> they were linked to too many other Pier objects.
>
> I'm now eliminating the custom mapping code by using Fuel substitution.
> The PRFile object was handled by saving the file contents in an instance
> variable of the mapping object. I've eliminated the custom mapping object.
> Now, how should the file contents be saved/restored, using the substitution
> feature?
>
> I'm thinking of substituting a proxy object, which somehow "becomes" a
> PRFile after deserialization.
>

you can implement :

YourPRFileProxy >> fuelAfterMaterialization
self become: self proxiedPRFile

Anyway, in next version would be nice to provide a faster way of doing
this, without become.


> Any other suggestions?
>
>
>

Re: [Pharo-project] Fuel bug: infinite recursion in "analyzer when:substituteBy:"

[Martin Dias]

Hi Yanni

This is a feature or a known issue, I am not sure yet!

In serialization, when you substitute an object by another, such
substitution(*) is then serialized using the same rules, so you might need
the guard condition to avoid the infinite loop.

This was the simpler implementation, but maybe it's undesirable or it's too
error prone. Do you think we should change this?

*: note the substitution can be not just an object but a graph of objects,
some of them able to be substituted too.

Martín


On Fri, Nov 25, 2011 at 6:20 PM, Yanni Chiu  wrote:

> The attached test case, reproduced below:
>
> | result |
> self analyzer
>when: [:x | FLPair = x class and: [x left isNil not] ]
>substituteBy: [:x | x copy left: $A ].
> result := self resultOfSerializeAndMaterializ**e:
>(FLPair new left: $a; right: $b; yourself).
> self assert: result left = $A.
>
> causes an infinite loop with IdentityDictionary (see stack trace at end).
>
> The similar test case #testTransientPairLeft, works because the guard
> condition is negated by the replacement (i.e. "x left isNil not" is made
> false by the replacement).
>
> The attached test case does not make "x left isNil not" false. It causes
> an infinite recursion, so use "ctrl-." to stop it before too long.
>
> Is this a bug, or the intention of the #when:substituteBy: feature? I've
> worked around it, by making sure to always have the substituted value fail
> the guard clause.
>
> --
> Yanni
>
>
>  stack trace 
> IdentityDictionary>>scanFor:
> IdentityDictionary(**HashedCollection)>>**findElementOrNil:
> IdentityDictionary(Dictionary)**>>at:ifAbsent:
> FLPluggableSubstitutionCluster**(FLSubstitutionCluster)>>add:**traceWith:
> FLAnalysis>>mapAndTrace:
> FLAnalysis>>run
> [:anObject | (FLAnalysis newWith: self firstInMapperChain root: anObject)
> run; yourself] in FLAnalyzer>>setDefaultAnalysis
> FLAnalyzer>>analysisFor:
> FLSerialization>>analysisStep
> FLSerialization>>run
> [:anObject :aStream | (FLSerialization
>root: anObject
>on: aStream
>analyzer: self analyzer) run; yourself] in FLSerializer>>**
> defaultSerialization
> [self nextPutSignatureOn: aStream.
>self nextPutVersionOn: aStream.
>^ self serializationFactory value: anObject value: aStream] in
> FLSerializer>>serialize:on:
> BlockClosure>>ensure:
> FLSerializer>>serialize:on:
> [:aStream | self serializer serialize: anObject on: aStream] in
> FLPluggableSubstitutionTest(**FLSerializationTest)>>**serialize:
> [:aStream | aValuable value: aStream binary] in FLFileStreamStrategy>>**
> writeStreamDo:
> [anotherBlock value: file] in MultiByteFileStream class(FileStream
> class)>>detectFile:do:
> BlockClosure>>ensure:
> MultiByteFileStream class(FileStream class)>>detectFile:do:
> MultiByteFileStream class(FileStream class)>>forceNewFileNamed:do:
>

Re: [Pharo-project] class rename and Fuel

[Martin Dias]

Hi Yanni,

On Fri, Nov 25, 2011 at 7:22 PM, Yanni Chiu  wrote:

> On 25/11/11 3:56 PM, Yanni Chiu wrote:
>
>>
>> Any ideas why Fuel is looking for OldClass, yet there are no instances
>> of OldClass in the de-serialized object graph.
>>
>
> There are no instances, but the class object itself is being held in
> various places in the Pier kernel.
>

This can be the problem. Can you update these references to point NewClass?

I want to support in fuel 1.8 some way of specifying that a class has been
renamed. Something like:

FLMaterializer newDefault
classRenames: { #OldClass -> #NewClass }
materializeFrom: aFile.

Would this solve your problem?

cheers,
Martín

Re: [Pharo-project] fuel and a factory

[Martin Dias]

Hi
We plan add some option to rename at materialization time, like:

FLMaterializer newDefault
   classRenames: { #Boolean -> #MBoolean };
   materializeFrom: aFile.

I guess we shouldn't only pay attention to rename the classes but also
rename the symbol: If you have a compiled method that references to
#Boolean, the literal has to be updated too.

Martín

On Sun, Nov 27, 2011 at 1:59 PM, Yanni Chiu  wrote:

> On 27/11/11 8:31 AM, Mariano Martinez Peck wrote:
>
>>
>>
>> On Sun, Nov 27, 2011 at 7:44 AM, Stéphane Ducasse
>> > > wrote:
>>
>>Just out of curiosity
>>is there a way to specify the class that should be used when loading
>>a fuel saved objects.
>>For example I would like to know if it is possible to save Booleans
>>and load them as instance of another class MBoolean
>>
>>
>> Not a clean out-of-the-box hook that I can imagine. In fact, it seems
>> exactly the same feature Yanni is missing:
>> http://forum.world.st/class-**rename-and-Fuel-td4108831.html
>> **.
>> Right?
>>
>
> Not the same problem, since it was the class object itself that was being
> referenced in a Pier security permissions object.
>
>
>

[Pharo-project] Prune stack serialization

[Martin Dias]

Hi folks

In Fuel, we serialize a block closure with its state, including its
outerContext. This enables to serialize a sorted collection with its
sortBlock:

| bytes result |
bytes := FLSerializer serializeToByteArray: (SortedCollection sortBlock:
[:a :b | a > b ]).
result := FLMaterializer materializeFromByteArray: bytes.
result
addAll: #(1 2 3);
 yourself.
---> a SortedCollection(3 2 1)

Here the problem: the byte array is huge! (800kb) because we are
serializing unneeded context for the sort block.

We wonder how to prune it and save time and space.

Thanks in advance
Martín

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

(I am working on Pharo 1.3)

On Mon, Dec 5, 2011 at 2:16 AM, Martin Dias  wrote:

> Hi,
>
> I get a SubscriptOutOfBounds error when I evaluate:
>
> | aStream zipStream |
>
> aStream := ByteArray new writeStream binary.
> zipStream := GZipWriteStream on: aStream.
> zipStream nextStringPut: ''.
> zipStream nextStringPut: ''.
> zipStream close.
> aStream close.
>  aStream := aStream contents readStream binary.
> zipStream := GZipReadStream on: aStream.
> zipStream nextString.
> zipStream nextString.
> zipStream close.
> aStream close.
>
> The error doesn't happen when I put just one string (instead of two).
>
> Do you see something wrong in the code? Should I report a bug?
>
> Thanks,
> Martín
>

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

I forgot to say that I am working on a recent Pharo 1.3.

 And it is not only GZipWriteStream but also ZipWriteStream
and ZLibWriteStream .

Martín

On Mon, Dec 5, 2011 at 10:47 AM, Martin Dias  wrote:

> Hi,
>
> I get a SubscriptOutOfBounds error when I evaluate:
>
> | aStream zipStream |
>
> aStream := ByteArray new writeStream binary.
> zipStream := GZipWriteStream on: aStream.
> zipStream nextStringPut: ''.
> zipStream nextStringPut: ''.
> zipStream close.
> aStream close.
>  aStream := aStream contents readStream binary.
> zipStream := GZipReadStream on: aStream.
> zipStream nextString.
> zipStream nextString.
> zipStream close.
> aStream close.
>
> The error doesn't happen when I put just one string (instead of two).
>
> Do you see something wrong in the code? Should I report a bug?
>
> Thanks,
> Martín
>

[Pharo-project] A possible bug in GZip streams

[Martin Dias]

Hi,

I get a SubscriptOutOfBounds error when I evaluate:

| aStream zipStream |

aStream := ByteArray new writeStream binary.
zipStream := GZipWriteStream on: aStream.
zipStream nextStringPut: ''.
zipStream nextStringPut: ''.
zipStream close.
aStream close.
 aStream := aStream contents readStream binary.
zipStream := GZipReadStream on: aStream.
zipStream nextString.
zipStream nextString.
zipStream close.
aStream close.

The error doesn't happen when I put just one string (instead of two).

Do you see something wrong in the code? Should I report a bug?

Thanks,
Martín

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

I forgot to say that I am working on a recent Pharo 1.3.

 And it is not only GZipWriteStream but also ZipWriteStream
and ZLibWriteStream .

Martín

On Mon, Dec 5, 2011 at 10:47 AM, Martin Dias  wrote:

> Hi,
>
> I get a SubscriptOutOfBounds error when I evaluate:
>
> | aStream zipStream |
>
> aStream := ByteArray new writeStream binary.
> zipStream := GZipWriteStream on: aStream.
> zipStream nextStringPut: ''.
> zipStream nextStringPut: ''.
> zipStream close.
> aStream close.
>  aStream := aStream contents readStream binary.
> zipStream := GZipReadStream on: aStream.
> zipStream nextString.
> zipStream nextString.
> zipStream close.
> aStream close.
>
> The error doesn't happen when I put just one string (instead of two).
>
> Do you see something wrong in the code? Should I report a bug?
>
> Thanks,
> Martín
>

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

Hi!


> I guess you could add a guard for the #nextInto: message send in
> GZipReadStream>>#nextString when the length is 0. But I haven't tried.
>
> Sven
>
>
I think the problem is that this method supposes n >= 1:

InflateStream>>
next: n into: buffer startingAt: startIndex
 "Read n objects into the given collection.
Return aCollection or a partial copy if less than
 n elements have been read."
| c numRead count |
numRead := 0.
 ["Force decompression if necessary"
(c := self next) == nil
 ifTrue:[^buffer copyFrom: 1 to: startIndex+numRead-1].
"Store the first value which provoked decompression"
 buffer at: startIndex + numRead put: c.
numRead := numRead + 1.
"After collection has been filled copy as many objects as possible"
 count := (readLimit - position) min: (n - numRead).
buffer
replaceFrom: startIndex + numRead
 to: startIndex + numRead + count - 1
with: collection
startingAt: position+1.
 position := position + count.
numRead := numRead + count.
numRead = n] whileFalse.
 ^buffer

Martin

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

On Thu, Dec 8, 2011 at 2:50 PM, Sven Van Caekenberghe  wrote:

>
> On 08 Dec 2011, at 18:43, Martin Dias wrote:
>
> > I think the problem is that this method supposes n >= 1:
>
> Yeah, that is why I propose a guard, maybe in this method
>
> n = 0 ifTrue: [ ^ buffer]
>

thanks, it fixes the issue (inserted at the beginning of InflateStream>>
next:into:startingAt:).

The same issue should happen sending
InflateStream>>readInto:startingAt:count:

I guess we should add some tests in CompressionTests package. I can do it
on a couple of days.



> but I have no time to analyze the consequences/implications.
>
> Sven
>
>
>
>

Re: [Pharo-project] A possible bug in GZip streams

[Martin Dias]

Hi Levente,

On Thu, Dec 8, 2011 at 5:44 PM, Levente Uzonyi  wrote:

> On Thu, 8 Dec 2011, Martin Dias wrote:
>
>  On Thu, Dec 8, 2011 at 2:50 PM, Sven Van Caekenberghe 
>> wrote:
>>
>>
>>> On 08 Dec 2011, at 18:43, Martin Dias wrote:
>>>
>>>  I think the problem is that this method supposes n >= 1:
>>>>
>>>
>>> Yeah, that is why I propose a guard, maybe in this method
>>>
>>> n = 0 ifTrue: [ ^ buffer]
>>>
>>>
>> thanks, it fixes the issue (inserted at the beginning of InflateStream>>
>> next:into:startingAt:).
>>
>> The same issue should happen sending
>> InflateStream>>readInto:**startingAt:count:
>>
>> I guess we should add some tests in CompressionTests package. I can do it
>> on a couple of days.
>>
>
> Yeah, that "fixes" the issue, but why didn't it fail before the "fix",
> when there was only one string written and read? ;)
>

I don't understand what you mean. But I found something strange when
debugging, maybe it's the heisenbug you mentioned before:

| aStream zipStream |
aStream := ByteArray new writeStream binary.
zipStream := GZipWriteStream on: aStream.
zipStream nextPut: 5.
zipStream close.
aStream close.

aStream := aStream contents readStream binary.
zipStream := GZipReadStream on: aStream.
self halt.
zipStream next inspect.
zipStream close.
aStream close.

When the #halt happens, I just press "Proceed". One hopes that #next
answers 5 but not nil as it actually answers. Do you have a link to the
mentioned heisenbug description?

Martín


>
>
> Levente
>
>
>
>>
>>
>>  but I have no time to analyze the consequences/implications.
>>>
>>> Sven
>>>
>>>
>>>
>>>
>>>
>>
>

Re: [Pharo-project] Cleaning up SmartRefStream use

[Martin Dias]

On Sun, Dec 11, 2011 at 12:54 PM, Mariano Martinez Peck <
marianop...@gmail.com> wrote:

>
>
> On Sun, Dec 11, 2011 at 4:35 PM, Stéphane Ducasse <
> stephane.duca...@inria.fr> wrote:
>
>>
>> On Dec 11, 2011, at 4:26 PM, Mariano Martinez Peck wrote:
>>
>> >
>> >
>> > On Sun, Dec 11, 2011 at 11:32 AM, Stéphane Ducasse <
>> stephane.duca...@inria.fr> wrote:
>> > Indeed this is fun I was doing the same :)
>> > Now I do not understand why some parts of the system uses dataStream,
>> others ReferenceStream and finally others SmartRefStream
>> > why not all of them use SmartRefStream.
>> >
>> > It is not that I like these guys, but their class comments are pretty
>> clear:
>> > - DataStream does NOT handle cycles, but hence it may be faster in
>> certain scenarios. I guess that for example, Monticello is sure that in its
>> definitions, there cannot be cycles.  In addition, it doesn't take into
>> account variables names/order and hence class reshape isn't supported.
>> > - ReferenceStream supports cycles but still doesn't support class
>> reshape.
>> > - SmartRefStream supports cycles and class reshape.
>> >
>> sounds good
>
>
> Yes it does. In fact, I would love also to have those things
> optional/configurable in Fuel: optional cycle detection and optional class
> reshape support :)
>

But I don't see any performance cost on "class reshape support", do you?


>
>
>> then we should be able to replace all the reference stream and smart by
>> fuel.
>>
>
> Yes, but Fuel is not completly ready to fully replace them. There are some
> hooks they provide that Fuel does not yet. But we will get there at some
> point.
>
>
>>
>> stef
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

[Pharo-project] Class rename on materialization

[Martin Dias]

Hi. We have recently discussed in a couple of threads about
serialization & class renames. I want to ask you for the following
case, because I'm not sure what should be the expected result.

Suppose we serialize:

Array
with: OldClass new
with: OldClass
with: #OldClass.

Then, we specify to the materializer that OldClass has been renamed to NewClass.

Now, should we expect that

result
first ---> a NewClass
second --> NewClass
third --> #NewClass

or...

result
third --> #OldClass

?

In other words, should a class rename affect to every symbol in the graph?


Thanks,
Martín

Re: [Pharo-project] Class rename on materialization

[Martin Dias]

On Sat, Dec 17, 2011 at 2:44 AM, Yanni Chiu  wrote:

> On 16/12/11 3:09 PM, Mariano Martinez Peck wrote:
>
>> so guys???  Should we only renames Classes/Traits/ClassPools/**Methods
>>  or
>> we directly rename all ocurrences of a symbol ?
>>
>
> I haven't any use cases to decide. If you want a guess, then I'd say leave
> the symbols unchanged, because it's simpler to implement. Since there's no
> data points, at this point, why do work that may prove to be unnecessary.
>
>
Well, so I guess we will change just the classes and traits, and not every
symbol.

I was in favor of renaming every symbol because the "class rename"
refactoring changes any method who mention the original name.

Thanks!
Martin

Re: [Pharo-project] Class rename on materialization

[Martin Dias]

On Sun, Dec 18, 2011 at 12:05 AM, Yanni Chiu  wrote:

> On 17/12/11 9:28 PM, Martin Dias wrote:
>
>>
>> Well, so I guess we will change just the classes and traits, and
>> not every symbol.
>>
>> I was in favor of renaming every symbol because the "class rename"
>> refactoring changes any method who mention the original name.
>>
>
> That's a reference to a class, by its global name, and it's the right
> thing to rename it when refactoring.
>
> Now I'm not sure what is meant by "renaming every symbol". Is there some
> structure in a compiled method (which holds a class name symbol) which is
> going to be renamed? Or, are you talking about any serialized symbol, which
> happened to be the same as the class to be renamed, would be renamed? Or
> are both these scenarios the same?
>

For example, in the method

Object >>
arrayWithSomething
^Array with: OldClass with: #OldClass

if you inspect Object >> #arrayWithSomething, in the literals you will see:
- for the first array element, an association #OldClass->OldClass, which is
an entry in Smalltalk globals. This association should be materialized as
#NewClass->NewClass.
- for the second array element, the symbol #OldClass. Here is my dubt: 1)
materialize as #NewSymbol or 2) as #OldSymbol?

1) I think in this case we want to materialize the method as:

arrayWithSomething
^Array with: NewClass with: #NewClass

But...

2) suppose that in the same file we serialized some other object (not the
above mentioned method) who points to the symbol #OldClass. Maybe we don't
want to rename the symbol in this case.

Re: [Pharo-project] Fuel serialzing not truncating files

[Martin Dias]

Hi Norbert,

Thank you, it was wrong. We will include your fix will in next version.

After releasing Fuel 1.7, we have moved from squeaksource to
http://ss3.gemstone.com/ss/Fuel.html, so the fix is there.

We always appreciate the feedback, thanks!

Martín

On Mon, Dec 19, 2011 at 3:47 PM, Norbert Hartl  wrote:

> Hi Martin, Mariano,
>
> I wanted to check the permissions on squeaksource for fuel but
> squeaksource is about to die. So I drop you a note here (because I don't
> know a better place).
> In FLSerializer>>serialize:toFileNamed:
>
> you should use
>
>StandardFileStream
>forceNewFileNamed:  aFilename
>...
>
> instead of
>
>StandardFileStream
>fileNamed:  aFilename
>...
>
> Yours does not truncate the file. If your model is shrinking then the file
> stays of same size with garbage trailing. I didn't test if this introduces
> read problems but I think truncating is the way to go.
>
> Norbert
>
>
>
>

Re: [Pharo-project] Is there an adhoc header field in Fuel?

[Martin Dias]

Hi!
We don't have unused fields, but there is no problem to add your own prefix:

FileStream newFileNamed: 'demoMetadata.fuel' do: [:aStream |
aStream binary.
 aStream nextStringPut: 'metadata=5'.
FLSerializer newDefault
serialize: 'stuff'
  on: aStream ].

FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream |
aStream binary.
 metadata := aStream nextString.
stuff := (FLMaterializer newDefault
materializeFrom: aStream) root ].

Is this ok for you?

Best regards,
Martin


On Thu, Feb 9, 2012 at 8:25 PM, Yanni Chiu  wrote:

> I'd like to store some metadata in a Fuel serialized file. I'd like to be
> able to read that metadata, without de-serializing the file. So, if there
> were a fixed header at the start of the file, with some "unused" fields,
> that would be ideal.
>
>
>

Re: [Pharo-project] Is there an adhoc header field in Fuel?

[Martin Dias]

Maybe is too much overhead, but it is also possible to serialize an object
that represents the metadata as a prefix:

FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream |
 aStream binary.
metadata := Dictionary with: #meta -> 5.
content := 'stuff'.
 FLSerializer newDefault
serialize: metadata on: aStream;
serialize: content on: aStream ].

FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream |
| aMaterializer |
aStream binary.
 aMaterializer := FLMaterializer newDefault.
metadata := (aMaterializer materializeFrom: aStream) root.
 content := (aMaterializer materializeFrom: aStream) root ].

Martin

On Fri, Feb 10, 2012 at 8:25 AM, Igor Stasenko  wrote:

> On 10 February 2012 10:19, Mariano Martinez Peck 
> wrote:
> >
> >>> All you need to do is to send it a stream. If the
> >>> stream is already at a certain position, it doesn't matter it will
> start
> >>> from that position. Say you have stream where you write something first
> >>> and
> >>> then you give it to Fuel to serialize. The stream would be at some
> >>> position
> >>> different than zero. Say, at position 10. Fuel will start serializing
> for
> >>> there. Say we have written 40 bytes.  Then, at materialization, when
> you
> >>> give it the stream, you have to be sure to give the stream starting at
> >>> the
> >>> position 10.
> >>
> >> However if you consider Fuel "the standard file format" - this would
> >> appear to be broken by people using random offsets.
> >
> >
> >
> > yes, sure. But I think its ok to fail from the serializer point of view.
> It
> > is YOUR responsability to provide the correct streams.
> >
> >
> >>>
> >>> Fuel will load EXACTLY 40 bytes and will stop. The stream can
> >>> continuefuel doesn't care and it will store exactly when it
> finishes
> >>> reading his own bytes.
> >>>
> >>> BTW, this is how some databases work. The save several serialized
> objects
> >>> in the same stream...
> >>>
> >>
> >> To reach for the moon... Perhaps Fuel "the standard file format" holding
> >> multiple streams might be formalized by tagging different each stream
> with a
> >> 'name' and/or a 'type' and/or 'deserializerMethod'.  The first two of
> these
> >> might be '#fuel -> #fuelDeserializer' and '#meta->#textDeserializer'.
> >
> >
> > I don't understand. Fuel holding multiple streams ? why? for what?
> >
> >>
> >>  The the default Fuel system can then automatically ignore 3rd-party
> data.
> >
> >
> > I want to make my point clear: Martin proposed an example that solves
> Yanni
> > problem. If you serialize with a stream with a certain position, then you
> > have to materialize it with its correct position. Dot. It is perfect that
> > the materializer is broken if you do not give him the correct stream.
> > Now, I DO agree that there could be another solution rather than adhoc,
> > supported by fuel so that BY DEFAULT some data could be set and ignored
> at
> > the same time...so that there is no manipulation of the srteam by the
> user.
> >
> >
> no i think your approach is fine. you store what you need, and don't
> care/intervene with other parts of stream,
> so if user wants to add some stuff before materialized data, he free to do
> so.
>
>
> --
> Best regards,
> Igor Stasenko.
>
>

Re: [Pharo-project] Is there an adhoc header field in Fuel?

[Martin Dias]

On Fri, Feb 10, 2012 at 2:45 PM, Yanni Chiu  wrote:

> On 10/02/12 10:24 AM, Martin Dias wrote:
>
>> Maybe is too much overhead, but it is also possible to serialize an
>> object that represents the metadata as a prefix:
>>
>> FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream |
>> aStream binary.
>> metadata := Dictionary with: #meta -> 5.
>> content := 'stuff'.
>> FLSerializer newDefault
>> serialize: metadata on: aStream;
>> serialize: content on: aStream ].
>>
>> FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream |
>> | aMaterializer |
>> aStream binary.
>> aMaterializer := FLMaterializer newDefault.
>> metadata := (aMaterializer materializeFrom: aStream) root.
>> content := (aMaterializer materializeFrom: aStream) root ].
>>
>
> I like this solution best, since it doesn't change the Fuel "file format".
>
> I added some helper methods to my code base to make this usage scenario
> more obvious:
>
> FLMaterializer>>**nextMaterializedRootFrom: sourceStream
>^ (self materializeFrom: sourceStream) root
>
> FLMaterializer>>**allMaterializedRootsFrom: sourceStream
>| contents |
>contents := OrderedCollection new.
>[ sourceStream atEnd ]
>whileFalse: [ contents add: (self nextMaterializedRootFrom:
> sourceStream) ].
>^ contents
>
>
> Sample usage:
>
>
> FileStream forceNewFileNamed: 'demoMetadata.fuel' do: [:aStream |
> | serializer |
> aStream binary.
> serializer := FLSerializer newDefault.
> (1 to: 5) do: [ :each | serializer serialize: (Dictionary with: each ->
> (each * each)) on: aStream ].
>
> ].
>
> FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream |
> | aMaterializer |
> aStream binary.
> aMaterializer := FLMaterializer newDefault.
> aMaterializer allMaterializedRootsFrom: aStream ].
>
>
>
> FileStream oldFileNamed: 'demoMetadata.fuel' do: [:aStream |
> | aMaterializer |
> aStream binary.
> aMaterializer := FLMaterializer newDefault.
> aMaterializer nextMaterializedRootFrom: aStream.
> aMaterializer nextMaterializedRootFrom: aStream.
> ].
>

Great. Maybe we should include #nextMaterializedRootFrom: to fuel.

Based on metadata you might decide to not materialize the "real" content.
Like:

metadata := (aMaterializer nextMaterializedRootFrom: aStream).
[ metadata isWhatINeed ] ifTrue: [
realContent := aMaterializer nextMaterializedRootFrom: aStream ].

Re: [Pharo-project] Is there an adhoc header field in Fuel?

[Martin Dias]

I think that then this deserves  at least a page in the documentation,
> right?
>
> +1
 I'll add something.


>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

[Pharo-project] Fuel - class loading issue when superclass changed inst vars

[Martin Dias]

Hello,

I'd like to discuss about a problem found when loading classes with Fuel
(for people who don't know it:
http://rmod.lille.inria.fr/web/pier/software/Fuel).

Fuel can serialize a class either as a global or as a "regular" object. In
the latter case, it saves the class with its compiled methods, including
their bytecodes. (This was demonstrated in
http://marianopeck.wordpress.com/2011/09/24/importing-and-exporting-packages-with-fuel/
)

Well, the issue I want to talk about happens when loading a class with a
method that uses an inherited instance variable. In a short example:

- define class A with instVars 'x'
- define class B with a method that uses 'x'.
- serialize B in a file
- remove B from system
- redefine A with instVars 'y x'
- materialize B from the file

-> the bytecodes of B's method, that should refer to 'x', instead refer
to 'y' (because inst vars are encoded in bytecodes just as indexes)

Of course, there are variations of this problem: the variable was whether
removed or renamed.


Some solutions:
1) detect the change in a superclass and then:
  a) just raise an error
  b) compile based on attached source
  c) decompile parse tree from bytecodes, fix it, and generate the method.
2) serialize the parse tree *instead* of bytecodes, and always generate the
method when materializing.

I am thinking on the use case of package sharing, like Parcels. Note that
since Fuel supports multiple strategies for serializing the same object, it
is not necessary to choose just one option to fit in every use case.

- I'm not sure, but I guess Parcels makes something like 1c).
- The alternative 1a) doesn't require any compiler (in a bootstrap image is
good)
- I am tempted to explore 2), but not sure if that makes sense.

I am curious about your opinions.

Best regards,
Martín

Re: [Pharo-project] Fuel - class loading issue when superclass changed inst vars

[Martin Dias]

Hi!

Mariano, yes, I forgot to say that B is subclass of A.

Henry, thanks a lot for sharing your knowledge about BOSS. Following my
example, the "backwards-compatability reader method" has to be defined in
A, right? Or do you specify it as a parameter to the class loader? I would
like to have automatic tolerance to simple layout changes like this.

Eliot, thanks for the correction. Rescan the methods seems to be a nice
solution, and has not dependency with compiler.

The migration service you describe makes me think, because we have recently
added migration support to Fuel; it works like:

aMaterializer
migrateClassNamed: #OldPoint
toClass: NewPoint
 variables: {('x' -> 'posX'). ('y' -> 'posY')}.

On one hand, in simple cases it should be more efficient than creating a
dictionary for each instance and then become them. On the other hand, in
more complex cases it's not enough and the dictionary could be a more
uniform solution. For example, if NewPoint has just a variable named
"coordinates", which points to an array with x and y as elements.

Martín

On Tue, Mar 6, 2012 at 3:27 PM, Eliot Miranda wrote:

>
>
> On Tue, Mar 6, 2012 at 3:05 AM, Henrik Johansen <
> henrik.s.johan...@veloxit.no> wrote:
>
>>
>> On Mar 6, 2012, at 12:03 PM, Henrik Johansen wrote:
>>
>> >>
>> >
>> > Parcels
>> > - Check a hash based on class layout vs equivalent hash for the stored
>> method's class (stored with the method).
>> > If different:
>> > 1) check for existence of a backwards-compatability reader method, hand
>> the old instance to it, and expect the old instance to be #become'd into
>> something current,
>> > 2) raise an error if no such method exists.
>> >
>> > Now, the main problem with this scheme is it's left as an exercise to
>> the user to come up with a procedure to ensure said backwards-compat reader
>> method stays up to date as additional changes are made.
>> >
>> > Is it a big problem?
>> > Depends on your tests, and programmer diligence at any given day.
>> >
>> > Added inst vars are not a big deal, as if you forget them, there will
>> usually be a nil #DNU somewhere down the line. (or you use lazy
>> initialization, and everything works as expected even for existing
>> instances)
>> >
>> > Removals/reorderings are a bigger issue, as the detection of failure
>> (inst vars in wrong slots) are often far removed from the source of
>> problems (forgetting to update the reader method),
>> > If seldomly used, it may even go unnoticed until the instance is next
>> saved, at which time you're in real trouble (especially if saved alongside
>> newly created ones, in which case there is no consistency).
>> >
>> > In general I think it's an ok scheme, but would like to see a solution
>> more resilient to user-error.
>> > How to achieve that is an interesting topic, which I haven't yet found
>> time to think through as thoroughly as I had wished/intended some months
>> ago. :(
>>
>> Errr, BOSS, not Parcels.
>>
>
> Right. What Parcels do is shape-change instances *and* rescan methods to
> fix up inst var offsets.  When a parcel is saved the "signature" of classes
> of instances there-in are saved so that the parcel contains all the inst
> var names of the class and its superclass.  If on load the superclass chain
> has a different set of inst vars then lost inst vars are omitted and added
> inst vars are nil in the materialized instances.  If a Parcel contains a
> full class (not just instances) that has methods, and the superclass chains
> inst vars have changed  then these methods are "rescanned" (disassembled,
> and reassembled, not using the compiler) and inst var offsets are fixed up,
> missing inst vars getting changed into undeclared variable references (I
> think; at least this is what *should* happen; its what happens when one
> removes an inst var that is still referenced from methods).
>
> What's missing in the parcel scheme is any hook to allow the user to
> process shape-changing instances with the values of lost inst vars in hand.
>  Presumably there will be cases when those values are essential to any
> schema migration.  Personally I would want to decouple schema migration and
> add it as a post-processing step, which would imply that the materializer
> would offer a service (materialize in a special mode) where it constructed
> a dictionary from materialized and shape-changed instance to in parcel
> state (e.g. an Array of inst var values as they occurred in the parcel).
>  Then the materialized instance could be migrated after the fact, with the
> default behaviour being analogous to what the system does now on class
> redefinition (values of deleted inst vars are lost, aded inst vars are nil).
>
> This reminds me of a bug with the ClassBuilder/RefactoringBrowser
> combination.  If one refactors pushing an inst var up to a superclass or
> down to subclasses, the values of te inst var in instances are lost because
> the class change is in fact done as two changes, a deletion followed by an
> addition.  Th

Re: [Pharo-project] GSoC ideas, 5 days to go!

[Martin Dias]

Hi! here is another project:

Description: Package management with Fuel
Technical details: Fuel is a general purpose binary serializer. It already
saves and loads classes without using a compiler. Package management has
additional challenges like check dependencies, run pre- and post-scripts,
override existing classes or methods, tolerate superclass shape changes,
run system validations, send notifications, clean uninstall, and others.
Provide integration with current tools like Monticello, Gofer and Metacello
would be good.
Benefit for student: Complete understanding of the life cycle of classes
and packages in the system.
Benefit for community: Pharo/Squeak users will have the alternative to load
classes without compilation. This would be useful in bootstrap image
experimentation. This approach has proved great utility in other Smalltalk
environments, for example in Parcels (VW Smalltalk).

Cheers
Martin

On Wed, Mar 7, 2012 at 11:53 AM, Carla F. Griggio
wrote:

> Yay! :D
> Tonight I'll write a nice project description then. And even if it's not
> elected for the GSoC program I'll do it at work sometime during this year :P
>
>
> On Wed, Mar 7, 2012 at 11:17 AM, Geoffroy Couprie 
> wrote:
>
>> The old (97) Excel file format is a weird file system acting as a
>> database. It is also used for doc, msi and cab files. The new format (xslx,
>> docx, etc) is just a zipped bunch of xml files, so it's quite easy to parse
>> :)
>>
>>
>> On Wed, Mar 7, 2012 at 3:05 PM, Carla F. Griggio > > wrote:
>>
>>> Great!
>>>
>>> That reminds me... would it be too hard to be able to export Excel files
>>> from Pharo? Or at least Open Office Spreadsheets?
>>>
>>> That could be a project idea too.
>>>
>>> On Wed, Mar 7, 2012 at 10:38 AM, Tudor Girba wrote:
>>>
 Hi,

 Good one, Nick. I can offer to be second mentor on this one as well.

 Cheers,
 Doru



 On Wed, Mar 7, 2012 at 2:32 PM, Nick Ager  wrote:
 > Hi Janko,
 >
 > Here you go: 'ePUB Output for Pier Books'
 >
 > --
 >
 > Name: ePUB Output for Pier Books
 > Level: Intermediate
 > Possible mentor: Nick Ager
 > Possible second mentor: Lukas Rengli
 >
 > Description
 > Pier [1,2] is an extensible object-oriented content management
 system that
 > includes a book authoring engine. The book authoring engine has been
 used to
 > document a number of Smalltalk based projects, most notable Seaside
 [3] as
 > well as Moose [4] and Pharo [5].
 >
 > The goal of this project is add ePUB [6] as an output format to Pier
 > enabling existing and future Pier books and other Pier content to be
 output
 > in a format readable by popular eReaders.
 >
 > [1] http://www.piercms.com/
 > [2] http://code.google.com/p/pier/
 > [3] http://book.seaside.st/
 > [4] http://www.themoosebook.org/
 > [5] http://book.pharo-project.org/
 > [6] http://en.wikipedia.org/wiki/EPUB
 >
 > Technical Details
 >
 > Content in Pier is parsed into a document tree that supports the
 visitor
 > pattern [7] for traversing the internal representation. Pier currently
 > supports a number of output formats such as HTML, plain text, RSS,
 wiki
 > text. These formats are generated by visitors. The existing visitors
 would
 > act as a template for creation of an ePUB visitor which would
 generate ePUB
 > compatible output.
 >
 > [7] http://en.wikipedia.org/wiki/Visitor_pattern
 >
 > Benefits to the Student
 > The project is well defined with a clear deliverable. The student
 will gain
 > an understanding of the internals of a powerful, well structured
 content
 > management system as well as the satisfaction of having a set of
 artefacts
 > in the format of the existing Pier book content translated into ePUB
 > format.
 >
 > Benefits to the Community
 > The Smalltalk community and others using the Pier book authoring
 engine will
 > gain ePUB as an output format. With an ebook output format authors
 have a
 > potential new revenue source, encouraging a virtuous cycle of
 increasing
 > documentation within and outside the community.
 >



 --
 www.tudorgirba.com

 "Every thing has its own flow"


>>>
>>
>

Re: [Pharo-project] No red on Jenkins!

[Martin Dias]

Now that I am less busy, I will look at fuel yellows!

Martin

On Mon, Mar 12, 2012 at 8:56 AM, Pavel Krivanek wrote:

> Cool!
>
> -- Pavel
>
> On Mon, Mar 12, 2012 at 11:41 AM, Marcus Denker 
> wrote:
> > All green and yellow!
> >
> >https://ci.lille.inria.fr/pharo/
> >
> > Now the next goal is to make the yellow ones green. This is especially
> important for the VM related tests.
> > ("is this vm good?")
> >
> > e.g.
> >
> >
> https://ci.lille.inria.fr/pharo/view/VM-dev/job/Cog-Mac-Cocoa%20Tests/
> >
> >
> >
> > --
> > Marcus Denker -- http://marcusdenker.de
> >
> >
>
>

[Pharo-project] Reliable method change detection

[Martin Dias]

Hi

In Fuel (bleeding edge), a compiled method can be serialized as a global.
Basically, we store the class name and selector, so loading it is
straightforward... this can be fine if we suppose there were not any change
in the system between save and load time. This can be a huge supposition,
but maybe it's acceptable in some uses, for example to "log" exceptions.

Well, actually we can discuss if this is a good approach, but anyway I am
interested on how to detect a BIG change, for example the one I show in the
following code (which produces a vm crash):

| byteArray aClass aClosure materializedClosure |

aClass := Object subclass: #A.
 aClass compileSilently: 'methodWithClosure  ^ [ 42 ] '.

aClosure := aClass new perform: #methodWithClosure.

byteArray := FLSerializer serializeToByteArray: aClosure.

aClass compileSilently: 'methodWithClosure  ^ 42'.

materializedClosure := FLMaterializer materializeFromByteArray: byteArray.
materializedClosure value.


What can we do to detect this change?
- generate a checksum with bytecodes
- just compare the size
- something else?

Thanks,
Martín

Re: [Pharo-project] Reliable method change detection

[Martin Dias]

Finally I tried with a bytecodes checksum to detect changes. I chose a
16-bit variant of Adler-32 (http://en.wikipedia.org/wiki/Adler-32). Was
easy and worked fine.

ByteArray >>
adler16from: start to: end
"Answer a checksum over the values of the specified interval, calculated
using Adler-16 algorithm."
 | a b |
a := 1.
b := 0.
 start to: end do: [ :index |
a := (a + (self at: index)) \\ 251.
b := (b + a) \\ 251. ].
 ^(b bitShift: 8) bitOr: a

CompiledMethod >>
bytecodesHash
"Answer a 16-bit checksum of the bytecodes."
 ^ self adler16from: self initialPC to: self endPC


cheers,
Martín

On Sat, Mar 17, 2012 at 2:17 AM, Martin Dias  wrote:

> Hi
>
> In Fuel (bleeding edge), a compiled method can be serialized as a global.
> Basically, we store the class name and selector, so loading it is
> straightforward... this can be fine if we suppose there were not any change
> in the system between save and load time. This can be a huge supposition,
> but maybe it's acceptable in some uses, for example to "log" exceptions.
>
> Well, actually we can discuss if this is a good approach, but anyway I am
> interested on how to detect a BIG change, for example the one I show in the
> following code (which produces a vm crash):
>
> | byteArray aClass aClosure materializedClosure |
>
> aClass := Object subclass: #A.
>  aClass compileSilently: 'methodWithClosure  ^ [ 42 ] '.
>
> aClosure := aClass new perform: #methodWithClosure.
>
> byteArray := FLSerializer serializeToByteArray: aClosure.
>
> aClass compileSilently: 'methodWithClosure  ^ 42'.
>
> materializedClosure := FLMaterializer materializeFromByteArray: byteArray.
> materializedClosure value.
>
>
> What can we do to detect this change?
> - generate a checksum with bytecodes
> - just compare the size
> - something else?
>
> Thanks,
> Martín
>

[Pharo-project] Weird change in Project class >> spawnNewProcess

[Martin Dias]

Hi

Debugging a Fuel test case, I discovered something I can't explain myself:
Project class>>#spawnNewProcess is replaced in the middle of the *first*
test run, between serialization and materialization.

This happens the *first time* I run the test in a *clean* Pharo 1.3 image,
after first time, the method doesn't change anymore. In 1.4 we don't have
to problem.

I logged the method sources, and actually there is a difference:

-- during serialization --
'spawnNewProcess
UIProcess := [[World doOneCycle.
Processor yield.
 false] *whileFalse.*
* nil*] newProcess priority: Processor userSchedulingPriority.
 UIProcess resume'

-- during materialization --
'spawnNewProcess

UIProcess := [
 [World doOneCycle.  Processor yield.  false] *whileFalse: [].*
* *] newProcess priority: Processor userSchedulingPriority.
 UIProcess resume'

-- system info --
Pharo1.3 - 13327 - 'Croquet Closure Cog VM [CoInterpreter
VMMaker-oscog-EstebanLorenzano.139]'



Do you know why this happens?

Thanks,
Martín

Re: [Pharo-project] Reliable method change detection

[Martin Dias]

On Wed, Mar 21, 2012 at 11:09 AM, Alexandre Bergel
wrote:

> This is interesting. If we have this method on Object, can it be used to
> keep track of side effects? It looks like to isn't it?
>

For doing that, you should access any object in a byte-indexed way,  I
don't know how to do that in the "image side".



>
> Alexandre
>
>
>
> Le 19 mars 2012 à 23:35, Martin Dias  a écrit :
>
> Finally I tried with a bytecodes checksum to detect changes. I chose a
> 16-bit variant of Adler-32 (http://en.wikipedia.org/wiki/Adler-32). Was
> easy and worked fine.
>
> ByteArray >>
> adler16from: start to: end
> "Answer a checksum over the values of the specified interval, calculated
> using Adler-16 algorithm."
>  | a b |
> a := 1.
> b := 0.
>  start to: end do: [ :index |
> a := (a + (self at: index)) \\ 251.
> b := (b + a) \\ 251. ].
>  ^(b bitShift: 8) bitOr: a
>
> CompiledMethod >>
> bytecodesHash
> "Answer a 16-bit checksum of the bytecodes."
>  ^ self adler16from: self initialPC to: self endPC
>
>
> cheers,
> Martín
>
> On Sat, Mar 17, 2012 at 2:17 AM, Martin Dias  wrote:
>
>> Hi
>>
>> In Fuel (bleeding edge), a compiled method can be serialized as a global.
>> Basically, we store the class name and selector, so loading it is
>> straightforward... this can be fine if we suppose there were not any change
>> in the system between save and load time. This can be a huge supposition,
>> but maybe it's acceptable in some uses, for example to "log" exceptions.
>>
>> Well, actually we can discuss if this is a good approach, but anyway I am
>> interested on how to detect a BIG change, for example the one I show in the
>> following code (which produces a vm crash):
>>
>> | byteArray aClass aClosure materializedClosure |
>>
>> aClass := Object subclass: #A.
>>  aClass compileSilently: 'methodWithClosure  ^ [ 42 ] '.
>>
>> aClosure := aClass new perform: #methodWithClosure.
>>
>> byteArray := FLSerializer serializeToByteArray: aClosure.
>>
>> aClass compileSilently: 'methodWithClosure  ^ 42'.
>>
>> materializedClosure := FLMaterializer materializeFromByteArray: byteArray.
>> materializedClosure value.
>>
>>
>> What can we do to detect this change?
>> - generate a checksum with bytecodes
>> - just compare the size
>> - something else?
>>
>> Thanks,
>> Martín
>>
>
>

Re: [Pharo-project] Reliable method change detection

[Martin Dias]

I discovered that String implements #crc16, which adapted for my needs (as
#crc16from:to:), it's faster than my Adler implementation, so I moved.

cheers,
Martín

On Wed, Mar 28, 2012 at 4:15 PM, Martin Dias  wrote:

>
>
> On Wed, Mar 21, 2012 at 11:09 AM, Alexandre Bergel <
> alexandre.ber...@me.com> wrote:
>
>> This is interesting. If we have this method on Object, can it be used to
>> keep track of side effects? It looks like to isn't it?
>>
>
> For doing that, you should access any object in a byte-indexed way,  I
> don't know how to do that in the "image side".
>
>
>
>>
>> Alexandre
>>
>>
>>
>> Le 19 mars 2012 à 23:35, Martin Dias  a écrit :
>>
>> Finally I tried with a bytecodes checksum to detect changes. I chose a
>> 16-bit variant of Adler-32 (http://en.wikipedia.org/wiki/Adler-32). Was
>> easy and worked fine.
>>
>> ByteArray >>
>> adler16from: start to: end
>> "Answer a checksum over the values of the specified interval, calculated
>> using Adler-16 algorithm."
>>  | a b |
>> a := 1.
>> b := 0.
>>  start to: end do: [ :index |
>> a := (a + (self at: index)) \\ 251.
>> b := (b + a) \\ 251. ].
>>  ^(b bitShift: 8) bitOr: a
>>
>> CompiledMethod >>
>> bytecodesHash
>> "Answer a 16-bit checksum of the bytecodes."
>>  ^ self adler16from: self initialPC to: self endPC
>>
>>
>> cheers,
>> Martín
>>
>> On Sat, Mar 17, 2012 at 2:17 AM, Martin Dias wrote:
>>
>>> Hi
>>>
>>> In Fuel (bleeding edge), a compiled method can be serialized as a
>>> global. Basically, we store the class name and selector, so loading it is
>>> straightforward... this can be fine if we suppose there were not any change
>>> in the system between save and load time. This can be a huge supposition,
>>> but maybe it's acceptable in some uses, for example to "log" exceptions.
>>>
>>> Well, actually we can discuss if this is a good approach, but anyway I
>>> am interested on how to detect a BIG change, for example the one I show in
>>> the following code (which produces a vm crash):
>>>
>>> | byteArray aClass aClosure materializedClosure |
>>>
>>> aClass := Object subclass: #A.
>>>  aClass compileSilently: 'methodWithClosure  ^ [ 42 ] '.
>>>
>>> aClosure := aClass new perform: #methodWithClosure.
>>>
>>> byteArray := FLSerializer serializeToByteArray: aClosure.
>>>
>>> aClass compileSilently: 'methodWithClosure  ^ 42'.
>>>
>>> materializedClosure := FLMaterializer materializeFromByteArray:
>>> byteArray.
>>> materializedClosure value.
>>>
>>>
>>> What can we do to detect this change?
>>> - generate a checksum with bytecodes
>>> - just compare the size
>>> - something else?
>>>
>>> Thanks,
>>> Martín
>>>
>>
>>
>

[Pharo-project] SMark

[Martin Dias]

Hi.

I like SMark, so I am evaluating how straightforward would be to move Fuel
benchmark suite. A couple of questions:

1) I had some undeclared references, no problem?
HeadlessSave class>>run: (RVMOperations is Undeclared)
ScriptStarter class>>transportersForFileOutMenu (Transporter is Undeclared)

2) Can I run benchmarks in headless mode using ScriptStarter? How can I try
HelloWorld script? I couldn't figure out.


Thanks,
Martín

Re: [Pharo-project] Fuel and Pharo Kernel

[Martin Dias]

Nice :)

FuelPackageLoader is a functional prototype but needs many improvements. I
am elaborating a proposal for GSoC, with features like:

basic:
- emulate ClassBuilder (implies replacing it for something less monolithic):
  - validating the class definition
  - modifying an existent class
  - recompiling methods (superclass change tolerance)
  - mutating existing instances to the new format
  - registering the class as a global
  - send notifications
  - more
- handle dependencies
- good migration support
- visual tool: integrate with Gofer or Monticello?

extra:
- run pre and post scripts (to avoid compilation the scripts should be
fully serialized, not just the source code)
- new extension operations: add trait, add variable.
- clean uninstall: if an extension method overwrote an existing one, then
it should be restored when package uninstalled (I think in VW is like this).
- ?

Some items require community discussion, especially the extra ones. Also,
I'm not sure what should be in the scope of GSoC.

Cheers,
Martín

On Tue, Apr 3, 2012 at 5:19 PM, Pavel Krivanek wrote:

> On Tue, Apr 3, 2012 at 7:36 PM, Mariano Martinez Peck
>  wrote:
> >
> >
> > On Tue, Apr 3, 2012 at 6:39 PM, Pavel Krivanek  >
> > wrote:
> >>
> >> Hi,
> >>
> >> I tried the Fuel with Pharo Kernel and... it works well.
> >
> >
> > I am really happy to hear that.
> >
> >>
> >> It needs only
> >> very few modifications (removing of some dependencies).
> >
> >
> > Right now I am extremelly busy, but in 2 ways I can spend some time in
> Fuel.
> > If you think that some of these dependencies removal could be applied
> > directly to Fuel core, please let us know.
>
> Hm, I used packages from the Fuel CI image. As I see some things were
> changed. For the start, Fuel extensions of classes CharacterBlock and
> MetacelloGoferLoad should not be in the basic package.
>
> >
> >>
> >> But the really
> >> great thing is that the FLPackageLoader seems to work quite well too.
> >
> >
> > Awesome.
> >
> >>
> >> I do strongly recommend to add Fuel very early in the development of
> >> 1.5 into the basic kernel packages.
> >
> >
> > Indeed. That was the idea for 1.4 but we didn't have time.
> >
> >>
> >> I can not wait to see a Pharo
> >> image without the Compiler ;-)
> >>
> >
> > In fact that was one of the things Stef had in mind when he decided to
> > support and push Fuel. Because RMOD is working in boostrap and related
> works
> > where you would like to boostrap and grow without a compiler ;)
>
> I know, but I expected that package loading in Fuel has much wider
> dependencies and that it will require a lot of next work.
>
> -- Pavel
>
>
> >
> >
> >>
> >> -- Pavel
> >>
> >
> >
> >
> > --
> > Mariano
> > http://marianopeck.wordpress.com
> >
>
>

Re: [Pharo-project] A test crashes (win32) cogvm

[Martin Dias]

I tried with latest Eliot vm and it's fine, no crash!


On Tue, Apr 3, 2012 at 2:24 PM, Mariano Martinez Peck  wrote:

>
>
> On Tue, Apr 3, 2012 at 7:14 PM, Igor Stasenko  wrote:
>
>> yes.. i had similar crashes.. and always involving the progress bar(s).
>> but i was unable to find a fully reproducible case. :(
>>
>>
> now you have it :)
> Thanks Martin for the report. I would like to help but until 14th I cannot
> do ANYTHING else than my paper :(
>
>
>>
>> On 3 April 2012 16:31, Martin Dias  wrote:
>> > Hi
>> >
>> > Almost 100% of the times (but not always) I run certain Fuel test, the
>> vm
>> > crashes. The code related to such test has not changed for a long time,
>> so I
>> > guess the problem appeared when I started to use the latest vm. I'd
>> thank a
>> > lot some help on this issue.
>> >
>> > To reproduce, evaluate:
>> >
>> > (FLBenchmarksTest selector: #testNumberOfResults) run
>> >
>> > in:
>> > - any of latest fuel images (por example,
>> >
>> https://ci.lille.inria.fr/pharo/job/Fuel%20Pharo%201.4/284/artifact/Fuel.zip
>> )
>> > - using latest cog vm for win32
>> > (https://ci.lille.inria.fr/pharo/job/Cog-Win32/21/artifact/Cog.zip)
>> >
>> >
>> > I attach dmps of crashes running in two computers, with both Pharo 1.3
>> and
>> > 1.4.
>> >
>> >
>> > Thanks in advance.
>> > Martín
>> >
>>
>>
>>
>> --
>> Best regards,
>> Igor Stasenko.
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>

[Pharo-project] Using stdout in Windows --> 'File stdout is closed'

[Martin Dias]

Hi

In my windows 7, I get an error when trying to write to "FileStream stdout"
in both Pharo 1.4 and 1.3 (revisions #14423 and #13320).

To reproduce,

- download:
https://ci.lille.inria.fr/pharo/job/Pharo%201.4/373/artifact/Pharo-1.4.zip

- evaluate:
FileStream stdout nextPutAll: 'a'.

Is this a known issue? I couldn't find it in issue tracker.

Thanks in advance.
Martín


PharoDebug.log
Description: Binary data

Re: [Pharo-project] Milestone reached :-)

[Martin Dias]

Nice!
I guess fuel needs for materialization only 50% of its methods and 2/3 of
its classes. In a future I can work on that if you think it's worthwhile.

Martín

On Sun, Apr 8, 2012 at 1:59 PM, Stéphane Ducasse
wrote:

> Simply gorgeous!
>
> On Apr 8, 2012, at 6:23 PM, Pavel Krivanek wrote:
>
> > Hi,
> >
> > I created an experimental CI build that loads Fuel into the Pharo
> > Kernel, unloads Compiler and uses an special launcher that expects a
> > binary package name as command line argument.
> >
> >
> https://ci.lille.inria.fr/pharo/view/Pharo-Kernel/job/Pharo%20Kernel%20Fuel/
> >
> > You can test it with something like (see attachment):
> >
> > CogVM PharoKernel-Fuel-1.4.image testingPackage.fuel
> >
> > Following code exports a fuel binary package from the image:
> >
> > FileStream forceNewFileNamed: 'testingPackage.fuel' do: [:aStream |
> >   aStream binary.
> >   FLPackageStore new storeOn: aStream packageNamed:
> 'TestingPackage'.].
> >
> > As you can see from the output of the testing class, there is no
> > Compiler class in the image.
> >
> > Cheers,
> > -- Pavel
> > 
>
>
>

Re: [Pharo-project] SMark

[Martin Dias]

Hi Stefan, thank you very much. I think it'll be useful for Fuel. Keep you
posted!

Best regards
Martín

On Thu, Apr 5, 2012 at 5:28 PM, Stefan Marr wrote:

> Hi Martin:
>
> On 04 Apr 2012, at 09:57, Stefan Marr wrote:
>
> > Hi Martin:
> >
> > On 03 Apr 2012, at 21:50, Martin Dias wrote:
> >
> >> I like SMark, so I am evaluating how straightforward would be to move
> Fuel benchmark suite. A couple of questions:
> >>
> >> 1) I had some undeclared references, no problem?
> >> HeadlessSave class>>run: (RVMOperations is Undeclared)
> >> ScriptStarter class>>transportersForFileOutMenu (Transporter is
> Undeclared)
> >
>
> I fixed these warnings and fixed a couple of minor issues, like missing
> initialization and use of deprecated stuff on Pharo 1.4.
>
> The configuration loads now without issues into Pharo 1.3, Pharo 1.4, and
> Squeak 4.3.
>
> There are two failing tests, but these are expected failures ;)
>
> Let me know if you have any other problems.
>
> Thanks
> Stefan
>
>
> --
> Stefan Marr
> Software Languages Lab
> Vrije Universiteit Brussel
> Pleinlaan 2 / B-1050 Brussels / Belgium
> http://soft.vub.ac.be/~smarr
> Phone: +32 2 629 2974
> Fax:   +32 2 629 3525
>
>

Re: [Pharo-project] [Vm-dev] Re: Using stdout in Windows --> 'File stdout is closed'

[Martin Dias]

On Tue, Apr 10, 2012 at 5:38 AM, Stefan Marr wrote:

>
> Hi:
>
> On 10 Apr 2012, at 02:58, Martin Sandiford wrote:
>
> > I get the same thing if I start Cog with no special command line, and
> > I think this seems kind of correct to me, as I'm not sure where the
> > output would go when there is no console attached.
> >
> >C:\foo> CogVM.exe test.image
>
> This works on unix/OSX, the stdout should just be the command line.
> Strange that it seems to work for you when you do a redirect.
>

Yes! that was I expected.

Anyway Martin's answer helped me, and discovered that:

   C:\foo> CogVM.exe test.image | more

is ok for me.

What I also didn't expect is that when running "-headless", still a window
is opened. I read something about evaluating "UIManager default
nonInteractiveManager" but I couldn't achieve to do so yet.

Thanks!


>
> Would guess that is a problem in the platform dependent code of the
> FilePlugin.
>
> Best regards
> Stefan
>
>
>
> --
> Stefan Marr
> Software Languages Lab
> Vrije Universiteit Brussel
> Pleinlaan 2 / B-1050 Brussels / Belgium
> http://soft.vub.ac.be/~smarr
> Phone: +32 2 629 2974
> Fax:   +32 2 629 3525
>
>

Re: [Pharo-project] Power of Fuel - Pharo-Kernel to Pharo-Core in 15 seconds

[Martin Dias]

Great! :D

Thanks Pavel, we have to work on these issues. I reported them in
http://code.google.com/p/fuel/issues/detail?id=143

It would be very nice to be full-time for the project during the
Winter (propaganda for my Google Summer of Code proposal ;) I'm on the
Southern hemisphere!)

BTW, we should find a better name than FuelPackageLoader!

Cheers
Martín

On Tue, Apr 10, 2012 at 10:49 AM, Edgar J. De Cleene  wrote:

>
>
>
> On 4/10/12 10:38 AM, "Pavel Krivanek"  wrote:
>
> >
> > With start of 1.5 we plan to integrate Fuel and split Morphic
> > packages. Then the show will begin :-)
> >
> > -- Pavel
>
> Yes, I want first row ticket !!
>
> And congrats
>
> Pavel
>
>
>
>

Re: [Pharo-project] [Vm-dev] Re: Using stdout in Windows --> 'File stdout is closed'

[Martin Dias]

On Tue, Apr 10, 2012 at 1:45 PM, Eliot Miranda wrote:

>
> Hi Martin,
>
> would you please try with both the Croquet.exe and the
> CroquetConsole.exe in
> http://www.mirandabanda.org/files/Cog/VM/VM.r2540/cogwin.zip?.  The
> CroquetConsole.exe is designed to be used from the command line; stupidly
> Windows has this distinction between console apps and windows apps, and
> this distinction affects standard i/o.  If CroquetCOnsole works then the
> implication is that Pharo needs also to build both VMs and the user needs
> to select the console VM for console use and the Windows VM for detatched
> use.
>

Oh, thanks, using CroquetConsole.exe it works fine.

Now, I realized that in DOS #cr does not work fine as "new line" so I
changed to #crlf... this was necessary also using cygwin and another bash
emulator.

But...  hmmm...

use a special vm + use a different newline character

 => I prefer to avoid that, just using the normal cogvm and redirect to a
file using ">" in command-line.

Thank you very much!


> HTH
> Eliot
>
> On Sun, Apr 8, 2012 at 4:31 PM, Martin Dias  wrote:
>
>> Hi
>>
>> In my windows 7, I get an error when trying to write to "FileStream
>> stdout" in both Pharo 1.4 and 1.3 (revisions #14423 and #13320).
>>
>> To reproduce,
>>
>> - download:
>> https://ci.lille.inria.fr/pharo/job/Pharo%201.4/373/artifact/Pharo-1.4.zip
>>
>> - evaluate:
>> FileStream stdout nextPutAll: 'a'.
>>
>> Is this a known issue? I couldn't find it in issue tracker.
>>
>> Thanks in advance.
>> Martín
>>
>
>
>
> --
> best,
> Eliot
>
>
>

Re: [Pharo-project] [Vm-dev] Re: Using stdout in Windows --> 'File stdout is closed'

[Martin Dias]

On Tue, Apr 10, 2012 at 8:41 PM, Eliot Miranda wrote:

>
>
>
> On Tue, Apr 10, 2012 at 4:35 PM, Martin Dias  wrote:
>
>>
>>
>>
>> On Tue, Apr 10, 2012 at 1:45 PM, Eliot Miranda 
>> wrote:
>>
>>>
>>> Hi Martin,
>>>
>>> would you please try with both the Croquet.exe and the
>>> CroquetConsole.exe in
>>> http://www.mirandabanda.org/files/Cog/VM/VM.r2540/cogwin.zip?.  The
>>> CroquetConsole.exe is designed to be used from the command line; stupidly
>>> Windows has this distinction between console apps and windows apps, and
>>> this distinction affects standard i/o.  If CroquetCOnsole works then the
>>> implication is that Pharo needs also to build both VMs and the user needs
>>> to select the console VM for console use and the Windows VM for detatched
>>> use.
>>>
>>
>> Oh, thanks, using CroquetConsole.exe it works fine.
>>
>> Now, I realized that in DOS #cr does not work fine as "new line" so I
>> changed to #crlf... this was necessary also using cygwin and another bash
>> emulator.
>>
>> But...  hmmm...
>>
>> use a special vm + use a different newline character
>>
>
> If you set its lineEndCOnvention correctly you shouldn't have to use a
> different character.  The special VM is forced by WIndows (and if you
> compare the bits you'll see they differ in only a few bytes).
>

Ok, so I will take the two alternatives into account, and I will see what
results better in the practice.


PS: I also rejected #crlf because I saw that Transcript in Pharo (at least
in 1.4 and 1.3) currently doesn't support it, so made me feel to better
don't use it. But maybe makes sense to report the issue and implement it.



>
>>  => I prefer to avoid that, just using the normal cogvm and redirect to a
>> file using ">" in command-line.
>>
>> Thank you very much!
>>
>>
>>> HTH
>>> Eliot
>>>
>>> On Sun, Apr 8, 2012 at 4:31 PM, Martin Dias wrote:
>>>
>>>> Hi
>>>>
>>>> In my windows 7, I get an error when trying to write to "FileStream
>>>> stdout" in both Pharo 1.4 and 1.3 (revisions #14423 and #13320).
>>>>
>>>> To reproduce,
>>>>
>>>> - download:
>>>>
>>>> https://ci.lille.inria.fr/pharo/job/Pharo%201.4/373/artifact/Pharo-1.4.zip
>>>>
>>>> - evaluate:
>>>> FileStream stdout nextPutAll: 'a'.
>>>>
>>>> Is this a known issue? I couldn't find it in issue tracker.
>>>>
>>>> Thanks in advance.
>>>> Martín
>>>>
>>>
>>>
>>>
>>> --
>>> best,
>>> Eliot
>>>
>>>
>>>
>>
>>
>
>
> --
> best,
> Eliot
>
>
>

Re: [Pharo-project] Fwd: Etoys, Arduino, Mac

[Martin Dias]

On Wed, Apr 11, 2012 at 3:55 AM, Stéphane Ducasse  wrote:

>
>
> Begin forwarded message:
>
> *From: *Ricardo Moran 
> *Subject: **Re: Etoys, Arduino, Mac*
> *Date: *April 11, 2012 12:08:16 AM GMT+02:00
> *To: *Torsten Sadowski 
> *Cc: *Stéphane Ducasse 
>
> Hi Torsten,
>
> I really appreciate what you're doing. Right now I'm super busy, though,
> because I just found out Physical Etoys is going to be included by default
> in all the laptops of the "Conectar Igualdad" project (which is like an
> argentinian version of OLPC),
>
>
Wow, nice! that means a lot of new potential users!


> so I have to hurry up and make a new version that works in windows and
> linux within the following 10 days. It's really *great* news but I'll have
> a lot of work to do (mostly testing) to make sure it works reliably.
>
> As for the packages install order, the following should work:
> 1) Connectors (already installed in Etoys image)
> 2) OSProcess
> 3) AST-Core
> 4) PhysicalEtoys
> 5) CTranslator
> 6) Arduino-Core
> 7) Arduino-Etoys
> 8) Arduino-Compiler
> 9) Arduino-EtoysCompiler
>
> Stef, some of these packages won't work for Pharo but the fixes done by
> Torsten should work. You can load them using the ConfigurationOfArduino and
> then manually fetching the Arduino-Compiler-TS.51.mcz from the Arduino repo.
>
> Thank you very much for your work, Torsten, as soon as I finish with this,
> I'll try to get someone to loan me a Mac so I can test and make a version
> with Mac support at last :)
>
> Cheers,
> Richo
>
>
> 2012/4/10 Torsten Sadowski 
>
>> Hello Richo, hello Stef,
>>
>> PhysicalEtoys should now work on a Mac. It is necessary to put the Image,
>> Arduino.app (Version 0023), an empty /Arduino/Sketches and possibly the VM
>> into a single directory. Then just add Connectors, OSProcess, CTranslator,
>> AST?, OmniBrowser?, Arduino-Core, Arduino-Etoys, Arduino-Compiler,
>> Arduino-EtoysCompiler and PhysicalEtoys from SqueakSource. I don't remember
>> the correct order and some stuff might be missing... Is there a
>> class/method search for SqueakSource?
>>
>> It would be nice if you could test what I have done and drop me a line if
>> it does not work.
>>
>> Cheers,
>> Torsten
>
>
>
>

Re: [Pharo-project] Power of Fuel - Pharo-Kernel to Pharo-Core in 15 seconds

[Martin Dias]

On Wed, Apr 11, 2012 at 2:22 PM, Craig Latta  wrote:

>
> > > BTW, we should find a better name than FuelPackageLoader!
> >
> > I propose Injector
>
>  Excellent name! ("Fuel injection" is a common automotive term in
> English.)
>

:) good names!

Well, I like Pump, but I vote for Injector also.

Pampa is already used by Fernando Olivero in Gaucho ;)

Barrel should be a candidate for the "fuel package", i.e. the file with the
classes, traits, extension methods, timestamps, etc. inside. We have to
think a name for this element also! Some synonyms / related words taken
from the dictionary:

barrel
cask
keg
tub
tun
tin
capsule
nitro

people choice?


>
>
> -C
>
> --
> Craig Latta
> www.netjam.org/resume
> +31   6 2757 7177
> + 1 415  287 3547
> + 1 510  282 7468
>
>
>
>

Re: [Pharo-project] Power of Fuel - Pharo-Kernel to Pharo-Core in 15 seconds

[Martin Dias]

so let use Injector!

for the package, we have:

barrel
can
tank
nitro
keg

Ok... probably I should go to sleep now instead of saying the following,
but... a tank can be a repository for nitros!


On Thu, Apr 12, 2012 at 8:46 AM, Ben Coman  wrote:

> Martin Dias wrote:
>
>> On Wed, Apr 11, 2012 at 2:22 PM, Craig Latta  wrote:
>>
>>
>>
>>> BTW, we should find a better name than FuelPackageLoader!
>>>>>
>>>>>
>>>> I propose Injector
>>>>
>>>>
>>> Excellent name! ("Fuel injection" is a common automotive term in
>>> English.)
>>>
>>>
>>>
>>
>> :) good names!
>>
>> Well, I like Pump, but I vote for Injector also.
>>
>> Pampa is already used by Fernando Olivero in Gaucho ;)
>>
>> Barrel should be a candidate for the "fuel package", i.e. the file with
>> the
>> classes, traits, extension methods, timestamps, etc. inside. We have to
>> think a name for this element also! Some synonyms / related words taken
>> from the dictionary:
>>
>> barrel
>> cask
>> keg
>> tub
>> tun
>> tin
>> capsule
>> nitro
>>
>> people choice?
>>
>>
>>
>>
>>> -C
>>>
>>> --
>>> Craig Latta
>>> www.netjam.org/resume
>>> +31   6 2757 7177
>>> + 1 415  287 3547
>>> + 1 510  282 7468
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
> +1 Injector - it aligns well with the task of injecting a package into the
> image. More than those other options, a fuel-can or fuel-tank is where you
> keep your fuel - but sometimes you can stretch an analogy too far.
>
>

Re: [Pharo-project] For Testing: 1.4 one-click

[Martin Dias]

Hi, I tested on Windows:

1) Pharo-1.4-one-click.bat points to a wrong image:

current:

"%~dp0\CogVM.exe" "%~dp0\Contents\Resources\pharo-1.4.image"

should be:

"%~dp0\CogVM.exe" "%~dp0\Contents\Resources\pharo-1.4-one-click.image"

after change, opened fine (with a console).


2) When I ran CogVM.exe by itself (not through .bat), it opened me the file
dialog like if it couldn't find the default image.


Martín

On Fri, Apr 13, 2012 at 9:24 AM, Stefan Marr wrote:

>
> On 13 Apr 2012, at 09:02, Marcus Denker wrote:
>
> > Hi,
> >
> > there is now an automatic build for the 1.4 one-click:
> >
> >
> https://ci.lille.inria.fr/pharo/view/Pharo%201.4/job/Pharo%201.4/lastSuccessfulBuild/artifact/Pharo-1.4-one-click.zip
> >
> > needs to be tested...
>
> $ sw_vers
> ProductName:Mac OS X
> ProductVersion: 10.6.8
> BuildVersion:   10K549
>
> $ uname -a
> Darwin minerva 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36
> PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386
>
> 8451 run, 8395 passes, 52 expected failures, 3 failures, 0 errors, 1
> unexpected passes
> Failures:
> MCClassDefinitionTest>>#testNotValidClassTraitComposition
> FileDirectoryTest>>#testRecursiveMatching
> MirrorPrimitiveTests>>#testMirrorSize
> MCClassDefinitionTest>>#testNotValidTraitComposition
>
> Errors:
>
>
>
> --
> Stefan Marr
> Software Languages Lab
> Vrije Universiteit Brussel
> Pleinlaan 2 / B-1050 Brussels / Belgium
> http://soft.vub.ac.be/~smarr
> Phone: +32 2 629 2974
> Fax:   +32 2 629 3525
>
>
>

[Pharo-project] SourceFiles

[Martin Dias]

Hi all

I vaguely remember some discussions about enhancing SourceFiles, something
about to reify a code manager, about to stop logging fileIns into .changes
file, and about to inspire the new mechanism on the VW's one.

Am I right?
There is consensus on this?
Do you think I could do it as part of Fuel Injector GSoC project?

Best regards
Martín

Re: [Pharo-project] SourceFiles

[Martin Dias]

Thanks!

On Tue, Apr 17, 2012 at 8:49 AM, Marcus Denker wrote:

>
> On Apr 17, 2012, at 9:12 AM, Martin Dias wrote:
>
> > Hi all
> >
> > I vaguely remember some discussions about enhancing SourceFiles,
> something about to reify a code manager, about to stop logging fileIns into
> .changes file, and about to inspire the new mechanism on the VW's one.
> >
> > Am I right?
>
> Yes.
>
> > There is consensus on this?
>
> About a better Interface (SourceManager): yes, consesus. About not having
> course and changes, I think everyone
> knows that it should be done eventually, but some argue that it works and
> is stable and changing it now would
> be problematic... there are good reasons for both views, I think.
>
> > Do you think I could do it as part of Fuel Injector GSoC project?
> >
>
> Yes, that could be an idea.
>
> --
> Marcus Denker -- http://marcusdenker.de
>
>
>

[Pharo-project] Nested progress bars

[Martin Dias]

Hi all.

Given the next code, could you explain me why nested progress bars are
not handled by my #on:do:?
How can I achieve that?

[
'main task'
displayProgressFrom: 0 to: 5
during: [ :bar |
0 to: 5 do: [:x | bar value: x.
(Delay forMilliseconds: 200) wait.

'nested task ' , x printString
displayProgressFrom: 5 to: 10
during: [ :bar2 |
5 to: 10 do: [:x2 | bar2 value: x2.
(Delay forMilliseconds: 200) wait] ] ] ]

]   on: ProgressInitiationException
do: [ :ex |
ex sendNotificationsTo: [ :min :max :curr |
self traceCr: curr ] ].


Thanks in advance.
Martín

Re: [Pharo-project] Nested progress bars

[Martin Dias]

I didn't mention that my question comes from trying and finding bugs
in the method BlockClosure>>silentlyValue, in Pharo 2.0 (should be in
1.4 also):

silentlyValue
"evaluates the receiver but avoiding progress bar notifications to show 
up."

^[ self value ]
on: ProgressInitiationException
do: [ :ex |
ex sendNotificationsTo: [ :min :max :curr |
self traceCr: min printString,'  ',max 
printString,'  ',curr printString; cr
].
].

> "; cr" has to be removed from code. After this easy fix, the
other bug makes visible: #silentlyValue is not that silently with
nested progress bars.  Try:

[
'main task'
displayProgressFrom: 0 to: 5
during: [ :bar |
0 to: 5 do: [:x | bar value: x.
(Delay forMilliseconds: 200) wait.

'nested task ' , x printString
displayProgressFrom: 5 to: 10
during: [ :bar2 |
5 to: 10 do: [:x2 | bar2 value: x2.
(Delay forMilliseconds: 200) wait] ] ] ].

] silentlyValue.


Cheers.
Martín


On Tue, May 8, 2012 at 12:28 AM, Martin Dias  wrote:
> Hi all.
>
> Given the next code, could you explain me why nested progress bars are
> not handled by my #on:do:?
> How can I achieve that?
>
> [
> 'main task'
>        displayProgressFrom: 0 to: 5
>        during: [ :bar |
>                0 to: 5 do: [:x | bar value: x.
>                (Delay forMilliseconds: 200) wait.
>
>                'nested task ' , x printString
>                        displayProgressFrom: 5 to: 10
>                        during: [ :bar2 |
>                                5 to: 10 do: [:x2 | bar2 value: x2.
>                                (Delay forMilliseconds: 200) wait] ] ] ]
>
> ]       on: ProgressInitiationException
>        do: [ :ex |
>                ex sendNotificationsTo: [ :min :max :curr |
>                        self traceCr: curr ] ].
>
>
> Thanks in advance.
> Martín

[Pharo-project] [ANN] Fuel Release Version 1.8

[Martin Dias]

Hi all,

We are proud to announce Fuel 1.8 with a lot of new features and
documentation. Of course, we don't stop working but it was time to
freeze a stable version.

Fuel is an open-source general-purpose object serialization framework
developed in Pharo Smalltalk environment. More information on our
website [1].

Fuel works out of the box in Pharo from 1.1 up to 2.0 (checked at
#20094). Please check our documentation [2] for complete installation
and use guides.

Below, there are some remarkable changes.

- Much more complete user guides.
- Improved API for customizing how graph is traced.
- FuelMetalevel: serialization of stuff like classes and compiled
methods moved to an optional package.
- Customize objects to be treated as globals.
- Migrations: declare at materialization time class and variables renames.
- Weak references properly managed. Thanks to Juan Vuletich and Levente.
- Thanks Pavel for helping us improving FuelPackageLoader with
wonderful ideas and bug reports.
- Serialization speed up on large graphs by using specialized
collections. Thanks Levente.
- Encoder and Decoder: new reifications that clarify the design.
Thanks Colin Putney.
- Optimized serialization of "clean" BlockClosures, which don't need
the whole stack of contexts. Thanks Juan Vuletich and Eliot Miranda.
- Added a clear error hierarchy.
- Debug facilities, including graph visualization using Roassal.
Thanks Alexandre and Doru.
- Many more new tests. Including those from extension packages, we
have almost 600 tests.
- Repository moved from 'http://www.squeaksource.com/Fuel' to
'http://ss3.gemstone.com/ss/Fuel'.

If you are a Fuel user we are not aware of, please let us know!

Best regards,
Martin and Mariano

[1]: http://rmod.lille.inria.fr/web/pier/software/Fuel
[2]: http://rmod.lille.inria.fr/web/pier/software/Fuel/Version1.8/Documentation

Re: [Pharo-project] [Pharo-users] [ANN] Fuel Release Version 1.8

[Martin Dias]

>
>> It has been long time since I looked into Pharo and licensing environment
>> (revitalising my interest now...), so I may be asking obvious questions but
>> what is the licencing model for Fuel?  (skimmed the mail and the web site
>> and only noted "open-sourced")
>>
>
> Thanks. I have updated the site. Now it says: "An open-source (MIT license)
> general-purpose object serialization framework developed in Pharo Smalltalk
> environment."

Thanks Jaroslaw, any feedback is welcome!

Re: [Pharo-project] [ANN] Fuel Release Version 1.8

[Martin Dias]

Hi Alexandre,

The screenshot looks great, thank you for Roassal, it's cool!

For those who may not know what are we talking about, what Alexandre
shows in the picture is an improvement to a new debugging tool for
Fuel, added in 1.8 version, which lets the user visualize the graph
that is being serialized. I called it FuelPreview, and you can play
with it if interested... more information on this guide:
http://rmod.lille.inria.fr/web/pier/software/Fuel/Version1.8/Documentation/Debugging

Fuel 1.8 still doesn't have the buttons bar of the screenshot.

Best regards,
Martín

On Tue, May 29, 2012 at 12:20 AM, Alexandre Bergel
 wrote:
>
> I've improved a bit the rendering with Roassal:
> Alexandre
>

Re: [Pharo-project] Nested progress bars

[Martin Dias]

On Tue, May 8, 2012 at 2:39 PM, Stéphane Ducasse
 wrote:
> Hi martin
>
> can you open a bug entry?

Okay, sorry for the delay. I opened two issues, 6004 and 6005.

Martín

Re: [Pharo-project] Fuel

[Martin Dias]

Hmm I didn't catch it

Martin

On Wed, Jun 6, 2012 at 4:47 PM, Stéphane Ducasse
 wrote:
> ok for me
> I'm studying PackageInfo and the rest to see what I can do.
>
>> mariano/martin
>>
>>        I would like to extract PakageInfo into self packageInfo in the fuel 
>> code.
>>
>> Yes, I like that the idea. I guess you are talking about FuelPackageLoader 
>> in #named: and #withPacakges: right?
>> What about calling it direclty #package?  so that we may be able to change 
>> it later to PackageInfo or RPackage or whatever?
>>
>>
>> Can I do it?
>>
>> Stef
>>
>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>
>

[Pharo-project] GSoC - Package management with Fuel

[Martin Dias]

Hi

This week has ended Google Summer of Code program 2012. I have to
thank Marcus and Carlos Ferro, my mentors, as well as Mariano,
Guille Polito, Camillo Bruni, and others, who collaborated in
different ways during these three months. Also thank, of course,
to Carla and Janko... and Google ;)

The main purpose of this mail is to summarize the work done for my
project, named "Package management with Fuel" [1][2].

Tools for managing code in Pharo export classes, traits, and methods
as series of source definitions, which then import by evaluating them
with the compiler. This is basically how FileOut/FileIn or Monticello
works. This is slow on large pieces of code, so if you want to install
a big package, you need a lot of time compiling it. Why should you
compile it again, given that it was all compiled when you exported the
package? The bytecodes will be the same again, you know... If you just
could load the Classes and Compiled Methods instead of compiling it
all again, you can save time.

With this goal in mind, I started working in FuelPackageLoader[4] (now
Tanker[3]), a prototype we wrote last year, which already exports and
imports packages without using any compiler. But it has two important
issues: it wasn't capable of export source code, and it wasn't able to
tolerate class-shape changes in hierarchies (ie, a change on inst vars
in a external superclass). So, we decided to start from scratch with a
new prototype, code name Carbono. The idea was to reify how FileOut
and FileIn are done, but storing binary information so that
compilation can be avoided.

In a nutshell, Carbono stores a package into two files:
- A sources file, imitating the traditional FileOut chunk format.
- A binary file, where uses Fuel[5] to serialize a series of code
definitions. Each of them knows the position in the sources file where
its source code counterpart is. For example, a class definition knows
the position of its comment (in the case it exists), and a method
definition knows the position of its source code.

A basic example, to export a couple of classes:

TAExport new
binariesOn: aBinaryStream
sourcesOn: aTextStream;
behaviors: {ClassA. ClassB};
methods: {};
initializeClasses: true;
run.

which then you can import with:

TAImport new
binariesFrom: aBinaryStream
sourcesFrom: aTextStream;
run.

Anyway, you can also choose to:
 - export only binaries or only sources
 - import only binaries (source code can be decompiled)
 - import only sources (via #fileIn)

Mariano Martinez Peck worked a lot on it, fixing bugs, and especially
optimizing it A LOT, so I hope you will enjoy his demonstration this
Monday on ESUG Innovation Awards. Seems that he could export Seaside,
Pier and Magritte, and import them in another image, with source code,
and just in a few seconds ;) Also, some ultra confidential WikiLeaks
revealed that he could export almost all Pharo packages, and import
them in a kernel image, all with source codes but without using the
compiler :)

Note1: Tolerating class-shape changes in hierarchies is not yet done,
but it should in some days. So far, the only strategy when a change is
detected will be to recompile methods, but it may be more
sophisticated in a not-so-far future.

Note2: Although finally it was not strictly necessary, Guillermo
Polito and me wrote tests for a new ClassBuilder. It is part of Slot,
a kind of reification of objects layout made by Toon Verwaest and
Camillo Bruni. It is too long to explain it here, but the thing is
that was not an easy job, and we hope it will be integrated in Pharo.

This is a work in progress, but you can install and try it on latest
Pharo 2.0, evaluating:

Gofer it
squeaksource3: 'PharoTaskForces';
package: 'Slot';
package: 'SlotTests';
load.
(Smalltalk at: #SlotVirus) perform: #spread.
Gofer it
squeaksource3: 'PharoTaskForces';
package: 'SlotsClassBuilderIntegration';
load.
Gofer new
smalltalkhubUser: 'marianopeck' project: 'Tanker';
package: 'ConfigurationOfTanker';
load.
(Smalltalk at: #ConfigurationOfTanker) loadBleedingEdge.
Gofer new
squeaksource: 'FuelExperiments';
package: 'Carbono';
package: 'CarbonoTests';
load.

Cheers,
Martín

[1]: http://gsoc2012.esug.org/projects/fuel-packages
[2]: 
https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2012/tinchodias/1
[3]: 
http://marianopeck.wordpress.com/2012/08/11/tanker-transporting-packages-with-fuel/
[4]: 
http://marianopeck.wordpress.com/2011/09/24/importing-and-exporting-packages-with-fuel/
[5]: http://rmod.lille.inria.fr/web/pier/software/Fuel

Re: [Pharo-project] Case study/ aside test for tanker

[Martin Dias]

how can I try?

Gofer it
squeaksource: 'MethodWrappers';
package: 'MethodWrappers4';
load.

?



On Fri, Nov 16, 2012 at 6:55 AM, Stéphane Ducasse  wrote:

> with the default system, trying to reload methodWrapper in a image with
> methodWrapper leads to an error:
> MethodStubWrpper cannot be recompiled.
>
> So I hope that the new classBuilder will be able to do it.
>
> Stef
>

Re: [Pharo-project] Zinc-WebSockets Live Demo

[Martin Dias]

I installed and played a bit, it's cool!!
 thanks
Martin

On Wed, Jan 9, 2013 at 2:30 PM, Sven Van Caekenberghe  wrote:
> Hi,
>
> Yesterday I deployed a live demo of Zinc-WebSockets
>
>   http://websocket.stfx.eu
>
> Since most proxies and load balancers, most notably Apache, don't handle the 
> WebSocket protocol very well, the only solution is to arrange for direct 
> connections to the Smalltalk image.
>
> Deploying and running this code in your own image is pretty easy:
>
>   ConfigurationOfZincHTTPComponents project latestVersion load: 'WebSocket'.
>
>   ZnServer startDefaultOn: 1701.
>   ZnWebSocketDelegate installExamplesInDefaultServer.
>
> Then open http://localhost:1701/ws-demo with your browser.
>
> Sven
>
> PS: You can read more about Zinc-WebSockets here
>
>   https://github.com/svenvc/docs/blob/master/zinc/zinc-websockets-paper.md
>
> PS: This demo, like the standard Zn and SSO demos run in the very latest 
> Pharo 2.0 version
>
>   http://zn.stfx.eu/status
>   http://sso.stfx.eu/status
>   http://websocket.stfx.eu:/status
>   http://websocket.stfx.eu:/ws-status-client
>
> --
> Sven Van Caekenberghe
> http://stfx.eu
> Smalltalk is the Red Pill
>
>