On Wed, Oct 14, 2015 at 12:09 PM, Robert Withers <robert.w.with...@gmail.com > wrote:
> > On 10/14/2015 11:01 AM, Mariano Martinez Peck wrote: > >> Robert, >> >> As far as I can remember, the problem of substitutions at >> materialziation time was because... as you may have read, Fuel first >> serializes "objects" and then the references. At materialization, it >> first creates the objects and then, on a second step, sets the >> references betwen objects. So the problem was WHERE to place the hook, >> at which point in time. If we do it just after objects were created, >> then the substitution code would NOT have access to any of the instVars >> of that object. If we do it AFTER objects creation and after objects >> references, then there is no easy way to "substitute" which doesn't >> involve #become: (because the graph is already constructed) And you know >> that #become: is still terrible slow because it scans full memory. That >> will change in Spur. >> > > The trick I learned from Gemstone is to use forwarding proxies, Well, Spur will/does something similar called lazy become. Basically it lets a forwarding pointer object and then takes advantage of next GC pass or whatever in order to resolve such proxy in lazy manner. > looking up into the FLobjectId dictionary (decoder>>objects?) when > stitching the references. When you copy the proxies on substitution, it > stitches normally at reference time. Yes, that could work. The problem is if you need instVars of the object you want to substitute. Imagine you have a class called Client and instVar 'age'. And you want to substitute Client with instances of OldClient if 'age' is > 10. At that point in time, the reference to the instVar 10 has not yet been filled. Yet, you need to replace the object at graph construction time. > > > As for Marea and Ghost, >> >> Ghost proxies paper: https://hal.inria.fr/hal-01081236/document >> Current Ghost repo: http://smalltalkhub.com/#!/~CAR/Ghost >> >> Marea paper: http://www.jot.fm/issues/issue_2013_01/article2.pdf >> Current repo: http://ss3.gemstone.com/ss/Marea.html >> >> And finally, my PhD thesis: >> https://tel.archives-ouvertes.fr/tel-00764991/document >> > > Nicely done, sir. I'll check them out, thankyou. > > Thanks, feel free to ask questions. > In Marea I needed custom clusters for my proxies because the serializer >> itself sends messages to the objects being serialized. My proxies would >> bring back graphs from a secondary memory. So if I was serializing a >> graph that had proxies already, I didn't want that. So I hooked my >> custom cluster for proxies that send only a few special messages to the >> proxy that these understand and answer rather than intercept those >> messages. >> >> As for how to extend Fuel for this, I recommend to check the code of >> Marea. See categories 'Marea-Serializers' and 'Marea-Proxies'. >> I have a Marea one click here: >> https://www.dropbox.com/sh/xp8jzyypmz0898j/AACRdHno6V7UfhaJ1ofTPPXva?dl=0 >> >> Cheers, >> > > thanks so much ^^ > Robert > > >> >> On Wed, Oct 14, 2015 at 11:40 AM, Robert Withers >> <robert.w.with...@gmail.com <mailto:robert.w.with...@gmail.com>> wrote: >> >> Good morning, Max. Thank you for the example. I got a little >> confused, between migrations and substitutions. My issue with no-arg >> blocks, I believe, is the inability to access my scope object to >> maintain the object tables. >> >> I'm attempting to write my own Materializer, Decoder and >> Materialization. At the moment, I'm just going to walk the graph, >> testing and do #becomes:. See how well that works when I get a test. >> >> It's really good to know about that other list. >> >> thanks so much ^^ >> Robert >> >> On 10/14/2015 04:15 AM, Max Leske wrote: >> >> BTW, there is a dedicated Fuel mailing list: >> pharo-f...@lists.gforge.inria.fr >> <mailto:pharo-f...@lists.gforge.inria.fr> >> <mailto:pharo-f...@lists.gforge.inria.fr >> <mailto:pharo-f...@lists.gforge.inria.fr>> >> >> Max >> >> >> On 14 Oct 2015, at 09:45, Max Leske <maxle...@gmail.com >> <mailto:maxle...@gmail.com> >> <mailto:maxle...@gmail.com <mailto:maxle...@gmail.com>>> >> wrote: >> >> >> On 14 Oct 2015, at 04:39, Robert Withers >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>> wrote: >> >> >> On 10/13/2015 09:43 PM, Mariano Martinez Peck wrote: >> >> >> >> On Tue, Oct 13, 2015 at 10:33 PM, Robert Withers >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>> wrote: >> >> Hi Mariano, >> >> This presents me with a big challenge, then. I >> read the docs and >> explored the code and the only other aspect not >> mentioned, beyond >> instance creation (#fuelNew #fuelNew:) and >> postMaterialization >> (#fuelAfterMaterialization), is migrations. >> However, migration only >> allows for instanceVar mappings, no code blocks. >> >> >> What do you mean that migrations only allows instVar >> mappings , and no >> code blocks? I mean, what do you mean by code blocks? >> >> >> Sounds to me like this (see FuelOutStackDebuAction): >> >> serializeTestFailureContext: aContext toFileNamed: aFilename >> | serializer | >> >> serializer := FLSerializer newDefault. >> self encodeDebugInformationOn: serializer. >> serializer addPostMaterializationAction: [ :materialization | >> Smalltalk tools debugger >> openOn: Processor activeProcess >> context: materialization root >> label: 'External stack' >> contents: nil >> fullView: false ]. >> >> serializer >> " use the sender context, generally the current context is not >> interesting" >> serialize: aContext >> toFileNamed: aFilename >> >> This stores a block in the serialization which is evaluated >> after >> materialization. The only requirement is that it’s a clean >> block (no >> closure!). >> >> We also support class renames. This is here: >> >> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/Migration?_s=Pvs4DYqPRjfEy8sg&_k=X6ltJu7FDSxumm4X&_n&22 >> >> Which kind of migration example you have in mind >> that would not be >> supported? An example would help. >> >> >> Well, my pics will demonstrate. I am interested in doing >> more than >> mappping ivars or a class rename. I want to do a total >> substitution, >> then a further substitution on the receiving, import side: >> >> Vat1: anObject (Class A) ---> On wire: desc (Descriptor) >> ---> Vat2: >> aProxy (Class FarERef) >> >> A desc is substituted for a PassByProxy object, then a >> FarERef is >> substituted for the desc. >> >> >> #fuelAccept: is a serialization side method. >> >> If Fuel supports substitution on serialization, I >> don't understand >> why no substitution support on materialization. >> >> >> There was a reason, which I cannot remember >> completely. Maybe Martin or >> Max can remember. >> >> >> It seems your focus was pickling to disk then back. My >> focus is >> distributed proxy graphs, which has different use cases. >> >> >> >> I am definitely going to use the world-class Fuel >> binary >> serialization system. However, I find myself >> needing to extend Fuel >> to support substitution on materialization. >> Perhaps the solution is >> a custom decoder. >> >> >> I have made custom clusters for example for my Ghost >> proxies of Marea >> system. It was a perfect example of how I could >> extent Fuel besides the >> common hooks. Fuel provides many places for >> extending , like clusters, >> analyzer, etc >> >> >> Right on, exactly! Could you tell me more about your >> Ghost proxies >> and Marea, please? As well, could you mention how you >> select a custom >> cluster on the serialization side? >> >> >> thanks so much ^^ >> Robert >> >> >> >> No, a bit more. It looks like I need a new >> FLSubstitutePointerObjectCluster, write them on >> serialization with >> the substitute, then do unsubstitution on >> materialization, since the >> cluster controls materialization and not the >> decoder. >> >> Does this approach seem sound to you, from a you >> know architecture >> and design approach? >> >> >> There was an issue. Hope other can remember. If not, >> I will try to >> explan what I remember tomorrow. >> >> >> thanks so much ^^ >> Robert >> >> On 10/13/2015 04:49 PM, Mariano Martinez Peck >> wrote: >> >> No, unfortunately, as far as I can remember, >> we do not have >> that. There >> are another hooks you may use but only in >> certain scenarios >> (#fuelNew, >> #fuelAfterMaterialization, global sends, >> etc). But everything is >> listed >> in >> >> http://rmod.inria.fr/web/software/Fuel/Version1.9/Documentation/CustomizingGraph >> so if you didn't find anything of help in >> there there are >> chances >> there isn't anything. >> >> Cheers, >> >> On Tue, Oct 13, 2015 at 5:30 PM, Robert Withers >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>>> wrote: >> >> Yes, I meant dynamic substitution on >> materialization, to >> use the >> correct terminology. >> >> thanks, >> Robert >> >> >> On 10/13/2015 11:40 AM, Max Leske wrote: >> >> >> On 13 Oct 2015, at 17:16, Robert >> Withers >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>>> wrote: >> >> Every extra source helps, thank >> you. I see how to do >> non-stream substitutions on >> materializations, but the >> documentation did not indicate a >> way to do non-stream >> substitutions on serialization. >> Is it possible? >> >> >> I don’t understand what you mean by >> “non-stream”. Could >> you give >> an example? >> >> >> thanks, >> Robert >> >> On 10/13/2015 09:00 AM, Mariano >> Martinez Peck wrote: >> >> Hi Robert, >> >> As for the documentation, >> you have LOTS of >> tests, you >> have the chapter >> Torsten pasted, you have >> this documentation: >> http://rmod.inria.fr/web/software/Fuel >> >> But also, as for internals, >> there is a journal >> paper we >> wrote: >> >> http://rmod.lille.inria.fr/archives/papers/Dias12a-SPE-Fuel.pdf >> >> Let us know how it goes, >> >> >> On Tue, Oct 13, 2015 at 6:00 >> AM, Torsten Bergmann >> <asta...@gmx.de >> <mailto:asta...@gmx.de> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de>> >> <mailto:asta...@gmx.de <mailto:asta...@gmx.de>> >> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de>>> >> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de>> >> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de> <mailto:asta...@gmx.de >> <mailto:asta...@gmx.de>>>>> wrote: >> >> Hi Robert, >> >> Also checkout the >> chapter on Fuel in Pharo >> Enterprise book: >> >> >> https://ci.inria.fr/pharo-contribution/view/Books/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/EnterprisePharo-A4.pdf >> >> Bye >> Torsten >> >> Gesendet: Dienstag, 13. Oktober 2015 um >> >> 09:44 Uhr >> >> Von: "Robert Withers" >> >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>>>> >> >> An: pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>> >> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org> >> <mailto:pharo-dev@lists.pharo.org >> <mailto:pharo-dev@lists.pharo.org>>>> >> >> Betreff: Re: [Pharo-dev] binary >> >> serialization >> >> >> Yes, I have to do object substitutions. >> >> Thanks >> for the link! >> >> >> thanks, >> Robert >> >> On 10/13/2015 03:43 AM, Max Leske wrote: >> >> >> On 13 Oct 2015, at 09:40, Robert Withers >> >> >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>>>> wrote: >> >> >> Sven and Torsten, that's a binary >> >> serialization library! It >> will take time to learn >> it and how to use >> mappers. >> >> >> What is the format; is it language >> >> neutral? >> >> >> For quick serialization you don’t >> >> need to do >> anything. It works >> for (almost) all >> objects. Only if you >> want to >> exclude things or >> treat some objects in a >> special way, you >> will need >> to do some stuff. >> >> >> Documentation: >> >> http://rmod.inria.fr/web/software/Fuel. >> >> >> >> >> thanks, >> Robert >> >> On 10/13/2015 01:21 AM, Sven Van >> >> Caekenberghe >> wrote: >> >> Yes, it is called FUEL and it is a >> >> standard >> part of the >> image. See FLSerializer >> and FLMaterializer. >> >> >> On 13 Oct 2015, at 06:59, Robert >> >> Withers >> >> <robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>> >> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com> >> <mailto:robert.w.with...@gmail.com >> <mailto:robert.w.with...@gmail.com>>>>> wrote: >> >> >> Does Pharo have stream classes to >> >> binary >> de/serialize an >> object, such that the >> protocol accepts an >> object as >> an argument and >> converts it to a >> byteArray? >> >> >> -- >> thanks, >> Robert >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> <Exporting Vat.jpg><Importing Vat.jpg> >> >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > -- Mariano http://marianopeck.wordpress.com
