I’ll make my first run soon on my first package. I’ll do one package extra each
time I run, building up the code. Each package will have its own <package
name>Transform method, per the examples. I’m not sure how to build these
methods, except to assume that rewriting the pharo base methods is never wrong.
I’ll file-in the resulting .st file to see what breaks. Then I’ll go back to
the package whose .st source is not loading completely and add additional fixes
(class keeps, base methods rewrites if needed and missing, method code body
replacements if needed) to its transform method until it loads completely on
the next run. That could take a while. Is that what you do in practice?
I’m assuming the code writer is taking into account inter-package dependencies
in order to get the load order right.
Shaping
From: christian.hai...@smalltalked-visuals.com
<christian.hai...@smalltalked-visuals.com>
Sent: Friday, 5 August, 2022 05:53
To: 'Any question about pharo is welcome' <pharo-users@lists.pharo.org>; 'Pharo
Development List' <pharo-...@lists.pharo.org>
Subject: [Pharo-users] Re: [Pharo-users]Porting from VW 8.3 to Pharo:
pdftalkPackageChanges
The full project in which the mentioned method used looks like this:
PDFtalkProject
^ProjectChange
name: #PDFtalk
source: ((OrderedCollection new)
add: (Package name: #Values);
add: (Bundle name: #PDFtalk);
add: (Package name: #'Values
Testing');
add: (Bundle name: #'PDFtalk
Testing');
add: (Package name: #'PDFtalk
Demonstrations');
yourself)
changes: self pdftalkPackageChanges
nameMapping: (NameMapping
keep: ((OrderedCollection new)
add:
#{Smalltalk.PDF};
add:
#{PostScript.PSDictionary};
add:
#{PDFtalk.PDFObject};
add:
#{PDFtalk.PDFArray};
add:
#{PDFtalk.PDFDictionary};
add:
#{PDFtalk.PDFStream};
add:
#{PDFtalk.PDFString};
add:
#{PDFtalk.PDFDate};
add:
#{PDFtalk.PDFTypeDefinition};
add:
#{PDFtalk.PDFEncoder};
yourself)
classToNames: ((Valuemap new)
add:
#{SubscriptOutOfBoundsError} -> #Error;
add:
#{NonIntegerIndexError} -> #Error;
add:
#{NotFoundError} -> #KeyNotFound;
add:
#{KeyNotFoundError} -> #KeyNotFound;
yourself)
namespaceToPrefixes: ((Valuemap
new)
add:
#{Smalltalk.PostScript} -> 'PS';
add:
#{Smalltalk.PDFtalk} -> 'Pt';
add:
#{PDFtalk.Fonts} -> 'PtF';
add:
#{PDFtalk.Fonts.OpenType} -> 'PtOT';
yourself))
In the #source: are the bundles and packages to be transformed. The #changes:
(your method) specify the transforms (PackageChange) for each package
explicitly. Only packages contain code and therefore, only packages need the
code transformations. Bundles are transformed without transformations. (Well,
the code for pre-, post- whatever blocks are transformed with the class name
mappings rules).
So, the mapping from packages to the corresponding PackageChange has to be
stated somehow. Using a dictionary (Valuemap) for this seems also natural. The
only change I might like is to use pragmas to tag the PackageChange returning
methods with “their” package like
<package: ‘Values Tools’> or so. Putting the package reference into the
PackageChange is not a good idea, because all those Objects need to be created
before your can find out which package is affected. (Ok, I am creating all
PackageChange objects too…).
Happy hacking,
Christian
Von: Shaping <shap...@uurda.org <mailto:shap...@uurda.org> >
Gesendet: Freitag, 5. August 2022 03:26
An: 'Any question about pharo is welcome' <pharo-users@lists.pharo.org
<mailto:pharo-users@lists.pharo.org> >; 'Pharo Development List'
<pharo-...@lists.pharo.org <mailto:pharo-...@lists.pharo.org> >
Betreff: [Pharo-users] Re: [Pharo-users]Porting from VW 8.3 to Pharo:
pdftalkPackageChanges
This method
pdftalkPackageChanges
^(Valuemap new)
add: 'Values' -> self ValuesTransform;
add: 'PostScript' -> self PostScriptTransform;
add: 'PDFtalk Basics' -> self
PDFtalkBasicsTransform;
add: 'PDFtalk Typing' -> self
PDFtalkTypingTransform;
add: 'PDFtalk Basic Objects' -> self
PDFtalkBasicObjectsTransform;
add: 'PDFtalk Streams' -> self
PDFtalkStreamsTransform;
add: 'PDFtalk Data Structures' -> self
PDFtalkDataStructuresTransform;
add: 'PDFtalk Parsing' -> self
PDFtalkParsingTransform;
add: 'PDFtalk Colour' -> self
PDFtalkColourTransform;
add: 'PostScript Fonts' -> self
PostScriptFontsTransform;
add: 'PostScript CIDInit' -> self
PostScriptCIDInitTransform;
add: 'PDFtalk Fonts Basics' -> self
PDFtalkFontsBasicsTransform;
add: 'PDFtalk Fonts Type1' -> self
PDFtalkFontsType1Transform;
add: 'PDFtalk Fonts OpenType' -> self
PDFtalkFontsOpenTypeTransform;
add: 'PDFtalk Fonts' -> self
PDFtalkFontsTransform;
add: 'PDFtalk Graphics' -> self
PDFtalkGraphicsTransform;
add: 'PDFtalk Graphics Operations' -> self
PDFtalkGraphicsOperationsTransform;
add: 'PDFtalk XObjects' -> self
PDFtalkXObjectsTransform;
add: 'PDFtalk Images' -> self
PDFtalkImagesTransform;
add: 'PDFtalk Files' -> self
PDFtalkFilesTransform;
add: 'PDFtalk Document' -> self
PDFtalkDocumentTransform;
add: 'PDFtalk Rendering' -> self
PDFtalkRenderingTransform;
add: 'PDFtalk Shading' -> self
PDFtalkShadingTransform;
add: 'PDFtalk Interactive Features' -> self
PDFtalkInteractiveFeaturesTransform;
add: 'PDFtalk Deploying' -> self
PDFtalkDeployingTransform;
add: 'Values Testing' -> self
ValuesTestingTransform;
add: 'PDFtalk test resources' -> self
PDFtalkTestResourcesTransform;
add: 'PostScript Testing' -> self
PostScriptTestingTransform;
add: 'PostScript CIDInit Testing' -> self
PostScriptCIDInitTestingTransform;
add: 'PDFtalk Fonts tests' -> self
PDFtalkFontsTestsTransform;
add: 'PDFtalk tests' -> self
PDFtalkTestsTransform;
add: 'PDFtalk Demonstrations' -> self
PDFtalkDemonstrationsTransform;
yourself
effectively looks like the head transform structure for a project, in this case
all the PDFtalk stuff, which includes Values and Postscript.
This is not exactly a bundle idea, is it? It’s project spread across
potentially many bundles and packages.
I’ll start coding my transform with a similar method, and work down toward the
details. Takes a bit to get used to all the correct, yet dangling VW methods
that are useless in VW, but which will become new code in the target image and
there no longer appear to be dangling (with syntax highlighting aberrations).
Odd looking but completely by design.
Shaping
From: Shaping <shap...@uurda.org <mailto:shap...@uurda.org> >
Sent: Thursday, 4 August, 2022 19:18
To: 'Any question about pharo is welcome' <pharo-users@lists.pharo.org
<mailto:pharo-users@lists.pharo.org> >
Subject: [Pharo-users] Re: [Pharo-users]Porting from VW 8.3 to Pharo: Pharo100
fileOutValues
I will look, although this kind of always available on demand thing is too
disruptive for me…
I know what you mean. I try be disciplined about it. I really like to be able
to fix typos, because there are almost always typos. But e-mail is okay.
So that little example is a test that shows how the transformation is done. It
converts just package Values to a Pharo-compatible file-in. My task is then to
queue a bunch of ProjectChange instances like this one:
SmalltalkTransform.Pharo100>>
ValuesProject
^ProjectChange
name: #Values
source: (Array with: (Package name: #Values))
changes: (Valuemap with: 'Values' -> self
ValuesTransform)
Exactly
but for my own packages. No bundles are transformed (just their contained
packages) because Pharo doesn’t have bundles.
Is that right?
No, bundles are handled. For real examples, you need to look at the PDFtalk
transforms.
Yes, Pharo does not have a concept of bundles (ordered aggregates of packages).
Instead it relies on a naming convention for packages. That convention is
honored in the fileout, so that packages will be partly grouped in Pharo
according to the category prefix.
For each VW-package, one Pharo package is created. A bundle itself is also
represented as Pharo package with one class About<bundlename> with class
methods for the metadata of the bundle, including a method giving you the
ordered list of component packages. So, all contents and metadata of packages
and bundles are transformed for Pharo. No code or info gets lost.
Okay.
Is method
ValuesTransform
^PackageChange
ignoredNames:
#(#{Smalltalk.GeneralBindingReference})
bridgeClasses: (Valuemap
with: #{Timestamp} ->
#DateAndTime
with: #{Smalltalk.ColorValue}
-> #Color)
localChanges: self valuesLocalTransform
extensions: (Array
with: (SystemClassChange
className:
#Color
instanceChanges: (Array with: (Add method: #asColorValue code:
#_ph_asColorValue)))
with: (SystemClassChange
className:
#TextStream
instanceChanges: (Array with: (Add method: #nextPutAllText: code:
#_ph_nextPutAllText:))))
written specifically for that package? I would think it applies to all
packages. I see some expected mappings like Timestamp to DateAndTime.
Yes, this method returns a PackageChange Value describing the transformations
needed to create the Pharo fileout for this specific package (inspect the
return value for the fully expanded Value). Methods exist with the same name
for other Smalltalks. Depending on the dialect (or version of a dialect), the
transforms are different. Squeak and Pharo are quite similar, because they
share a common history, but VA or Gemstone need quite different transforms.
So, in general, for each package, there is one such method/Value for each
target Smalltalk/version.
Okay.
I do not dare to extract commonalities before the machinery is really robust
and stable. For now everything is neatly separate and self-contained (and
probably it will stay that way, although there are lots of duplications).
The mapping of class names is the responsibility of the enclosing ProjectChange
Value where you define the list of source bundles/packages to transform, the
PackageChanges for all packages and the mapping of “global” names.
(The bridge classes above are no renames, but a subclass relationship (is-a) to
avoid renamings. The new class Timestamp will be created as subclass of
DateAndTime which has almost the same semantics. Therefore, I can still use
Timestamp which will be basically a DateAndTime now.)
Okay.
There is still a technical challenge here. Currently, a ProjectChange need to
include all prerequisites (Values is part of the PDFtalk project and will be
transformed with it). A ProjectWriter, which coordinates the transform, keeps
track of the mappings when they are created (either explicitly or through a
namespace renaming – see implementers of #PDFtalkProject).
I would like to have this more modular: the mappings from the Values
transformation should be persistently saved, so that other transformation
projects can just use them, instead of including the sources into one own
project.
For this, I need to have renamings local to a package (where they first occur),
not global on the project level.
Right.
For Values and Values Tests and Values Tools this works, because there are no
mappings in the Values package.
What about conversion of VW arrays to Pharo literal arrays? How is that done?
(I think you mean dynamic arrays like {1. ‘abc’ size. 42} in which evaluation
happens (in contrast to literal things which can be resolved already by the
compiler).
Yes, dynamic arrays.
Not! Since a while, VW also has dynamic arrays, but not in VW 8.3 – the last
publicly available version.
Okay, I was wondering when that would happen.
I will not shut out those users, because “open-source” would be quite absurd,
if it is only available for paying customers.
In 8.3, the compiler does not accept that syntax and therefore, there is no
easy way to represent this in replacement code.
So, no. It is not possible until Cincom releases an public version which can
handle that.
Okay.
I recall that one of the Smalltalks (I don’t recall which) had Stream semantics
differing from VW’s.
… I just checked. VW’s #upTo: method includes the object and leaves the index
after it, and Pharo’s excludes the object and leaves the index at the object.
So that is some major breakage if we don’t correct it. Can it be done
automatically?
Yes, these are the usual porting challenges and exactly the reason why this
library exists :). Thank you for the question :).
Yes, the stream semantics need to be fixed. The idea is that a set of
transforms for this issue can be reused by others.
Okay.
>> valuesLocalTransform
has lots of juicy bits. But this doesn’t look very simple. We can’t just
replace an old method with a new one. We also have to write the new one to
tweak how the indices are used in #upTo:, and make sure that new method gets
filed-in as well into the Pharo target image. Or, we have to do this kind of
change manually.
Naa, it’s very easy, I think :).
A PackageChange specifies transforms for classes used in the package
(#localChanges) and #extensions for system classes of the target. For a class,
you can have a ClassChange describing the changes to instance or class methods.
A MethodChange has 4 subclasses for:
- Ignore – don’t write this method to the target
- Add – add this new method (not in the source system) to the target
- Replace – replace the body of this method with other code
- Rewrite – rewrite the method source using a rewrite rule.
Add and Replace need the target code.
Add then always involves a new name for a method in the target. Replaces use
an old name in the target with a new code body.
This is stored in another method with a derived name like #_ph_upTo: . The
method name is not important, because only the body of the method is used. But
the name should not be used in the source – it is just a holder for the
replacement code. These methods live in the specific [<Smalltalk> Fileout
<Package>] package.
Okay.
There are lots of working(!) examples for all of those in the PDFtalk transform
project.
This bit
(SystemClassChange
className:
#Color
instanceChanges: (Array with: (Add method: #asColorValue code:
#_ph_asColorValue)))
is replacing #asColorValue with #_ph_asColorValue because some special
Pharo-color conversion needs to happen. But how does #_ph_asColorValue get
defined? It’s neither in VW nor in Pharo 10.
You got bitten by the old version of [Pharo Fileout Values]. Please load [Pharo
Fileout PDFtalk]. There, the methods exist.
Yes, I see it now.
Ok. I don’t have a virgin image. I have a very non-virgin image, about 27
years of development I’m trying to port to Pharo. I don’t yet have a specific
interest in the PDFtalk, though I do see a need for PDF generation later, and
will probably revisit that. For now, I just want my own stuff to run in Pharo.
Virgin image just means that you don’t need anything else. You can safely load
it in you favorite special images :).
I would load PDFtalk, although technically you don’t need to (all the
extensions to PDFtalk would be unloadable, but that doesn’t affect Values).
Okay.
is:
Load {Values Project] bundle
Load {PDFtalk Project} bundle
Load {Smalltalk Transform Project} bundle
Load [Pharo Fileout PDFtalk] package
Save, done
Okay, so do I understand correctly that I need to include the PDFtalk stuff
even if I’m not interested in PDFtalk, because that’s where a lot of the
Smalltalk transformation machinery lives? Or is the PDFtalk just being used as
an example for how to do a massive transformation? Or Both?
No, the transformation machinery is fully independent of PDFtalk. I just tried
it. The dependencies are in the specific [Pharo Fileout PDFtalk] package, since
I have already quite a few replacement methods which are extensions to PDFtalk
classes.
Okay.
PDFtalk is the focus of the project and therefore all issues are solved first
with this library in mind. Therefore, bundles and namespaces are handled, for
example. When you study the more interesting transformations for PDFtalk, it
would be a shame not to be able to browse the methods and classes involved.
So, PDFtalk is the real world reference example.
And Values is the simplest example.
To transform Values do: “Pharo100 fileOutValues”
The [Pharo Fileout PDFtalk] package includes the latest Values transformations.
I am thinking about a better modularization…
Also, the wiki is a bit out of control. It really needs some restructuring.
In the cites wiki page, there is a link to a blog where I record the changes.
This might be informative.
2. Port the Values package. This is easy, since no namespaces are involved.
This first instruction after VW package setup says to port the contents of the
Values package from VW to Pharo. Do you mean manually? Probably not.
No, no. This has been finished in March.
For each dialect, I have a GitHub repository where I release important
versions: <https://github.com/PortingPDFtalk/PharoPDFtalk>
https://github.com/PortingPDFtalk/PharoPDFtalk. You can find the working port
as first release “
<https://github.com/PortingPDFtalk/PharoPDFtalk/releases/tag/1.3.0.0> Working
version“. There you can download the ported Values fileout with the exact
description with which versions of what it was created.
This should be a good starting example.
Why do I need any new code installed in Pharo before I begin the
transformation, if I’m transforming code from VW to pharo? I’m not
understanding the basic constraints of the problem, even when the detailed
steps are clear.
No, no. You don’t need anything on the Pharo side. The fileouts on GitHub are
the end products of a transformation for people who don’t use VisualWorks, but
want to use Values in Pharo. Or help with PDFtalk by fixing some issues, so
that I can write the transformations.
I’ve done these steps so far:
1. Went to <https://github.com/PortingPDFtalk/PharoPDFtalk>
https://github.com/PortingPDFtalk/PharoPDFtalk.
Mistake :).
You don’t want to look at the unfinished product of the current version of
thincomplete transformations for PDFtalk.
Do you mean “finished” here? Isn’t that file-in the finished result?
I thought the above links was the currently finished result (as good as it can
be until the rest of the bugs are goine and tests all run).
Yes, that was confusing. That’s why I had the early impression that Values was
somehow apart of the transformation machinery.
See https://wiki.pdftalk.de/doku.php?id=stateoftheport#pharo-10-0:
Instead, you want to look at the unfinished product of the transformations of
your projects :).
2. Saved down PDFtalk.Pharo100.st into <my Pharo 10 image
directory>/pharo-local/Smalltalk-Transformation. (I figured that was a good
place to save it. If anyone disagrees, or has a better or more conventional
idea about where files should be saved, please say so. I setup a Pharo Git
repo and played with it briefly for the first time yesterday. I’ve used Pharo
off an on for 16 years, but this is the first time I’m making a serious effort
to manage source, and not throw away what I’m working on.)
3. Filed-in PDFtalk.Pharo100.st. This went on for about 7 minutes or so. I
have a hundreds if not over a thousand classes showing in Epicea. Is there
anyway to get Epicea to give me a count of changes with a time-range filter?
4. Deleted (forgot) yesterday’s, old Main practice-repo from both the image
and the drive, and made a new one. I need to add to Main all the packages I
just filed-in, but I don’t see an efficient way to do that. I would like to
use the Add Package button, but this gives a filtered list of available
packages. I can filter subgroups, and then individually select each of the
checkboxes to the left of each package (there is no Ctrl-A [select all] option
here, which seems to be a strange omission given the potentially large number
of packages involved). I see lots of prefixes for the classes just loaded. I
could easily miss something if I filter/select/add one prefix-group at a time.
Is there an easier way? Over in Epicea I don’t see a way to push the loaded
items listed to a specific repo. The first thought I have is to select all
filed-in code artifacts by datetime span. I did that and saved it as an .ombu
file (I have no idea what that is). I don’t see a way to import that .ombu
file into repo Main’s “Working copy” window. It must be easy, but I don’t see
it. Please suggest the best way.
I’d like to know as well – I am not quite familiar with the source management
concepts in Pharo.
I asked in Discord. I don’t understand why stuff like this is missing. The
only conclusion I can draw is that no one does huge file-ins (but you do).
It is planned to generate Tonel output in the future, but for now I feel safer
with the traditional fileout where I can have doIts. I am not sure how Tonel
reacts to crippled sources, which are normal during the development of the
transformations.
Shaping
From: christian.haider <christian.hai...@smalltalked-visuals.com
<mailto:christian.hai...@smalltalked-visuals.com> >
Sent: Wednesday, 22 June, 2022 05:42
To: pharo-users@lists.pharo.org <mailto:pharo-users@lists.pharo.org>
Subject: [Pharo-users] [PDFtalk] second fileOut for Squeak and Pharo
With help from the community some issues were fixed which improved the test
statistics nicely.
Check it out:
https://wiki.pdftalk.de/doku.php?id=portingblog#second_pdftalk_fileout_for_squeak_and_pharo
Thanks to everybody involved!
Happy hacking,
Christian
