2015-05-09 23:21 GMT+02:00 Tudor Girba <tu...@tudorgirba.com
<mailto:tu...@tudorgirba.com>>:
I do not think there are many people around here that would think
that it is irrelevant if the Pharo VM can be developed in Pharo or
not. Of course, it is important.
So, the discussion should not go to challenge this direction, but
rather in you telling us the use cases that you need supported.
Please note that I did not say which exact code and how it should
look like. I would be interested in learning about the use cases
you have. I am quite certain that there are a number of ways to
support them and when we work on GT it would be useful to have
your use cases on our table.
Well I need many lines to explain each point and there are many... I
can talk here about a few points. Then I will deal with Esteban for
most of them because it is difficult to explain without an interactive
discussion.
Let me explain the use cases for the Transcript for example. The
issues in Pharo are:
- The Transcript does not show the stream as it is printed.
- The Transcript does not inherit from Stream and thus cannot print
with all the methods implemented in Stream.
- The Transcript does not allow the user to decorate the text with
bold, italic or colors.
/Usecase 1: Debug printing methods:/ In the VM you have debug printing
methods, for example, to print the call stack. These methods are used
from the VM simulator, to output the string in the Transcript, and in
gdb, to ouput the string in the commandline. The commandline
(FileStream stdout in Pharo) and the Squeak Transcript have the same
behavior. In Pharo, the Transcript does not inherit from Stream so you
can't use the required stream methods to print the debug printing
method on the Transcript. In addition, some printing methods print a
lot of things and it is important to show the stream as it is printed.
For this use-case, we want to keep the smallest difference between the
gdb/commadline behavior and the VM simulator/Transcript behavior. If
you implement advanced tooling in GT, you therefore need to implement
gdb extensions (and lldb extensions because some of us use lldb
instead of gdb) and maintain them. I don't think this is a solution.
/Usecase 2: CCode generation debugging:/ The CCodeGenerator or Slang
translator translates Slang code into C code. Sometimes there is a
bug. To debug, instead of generating the faulty C method into an
external C file, we print only the faulty C method in the Transcript.
Again, we want to keep the lowest difference between the real usecase
(printing on the C file) and the debug usecase (printing on the
Transcript). In Squeak the FileStream and the Transcript are both
Stream, everything works as expected. In Pharo the Transcript has not
the expected behavior. Again the method can be long, you can have to
wait several seconds, so you'd like the transcript to show the stream
as you print it.
/Usecase 3: VM simulation:/ Simulating the VM is quite slow,
especially the machine code execution simulation. During the
simulation process, the UI is non interactive and shows only every
while what the simulator is doing in the Transcript. It is important
as sometimes when debugging with a test at each machine code
instruction it could take several hours before the UI is interactive
again and you want to know what is going on. I don't complain that it
takes several hours because the alternatives usually require days of
debugging and we can launch the VM simulator overnight. In Pharo this
does not work as expected.
/Usecase 4: In-image machine-code compilation:/ While working in the
JIT compiler, sometimes the machine code generated for a bytecoded
method is faulty. A common way of debugging it is to print the machine
code instructions of the machine code version of the method in the
Transcript. It can take a while to print, so it is important to have
the Transcript showing the text as it prints. Then, the easiest way of
debugging is to look at the machine code and understand what is wrong.
For this purpose, we add text decoration to color jump addresses or
the instructions where the instruction pointer was when the VM
crashed. Then, in squeak, we can easily copy the decorated text to a
workspace and generate a new version of the machine code method and
compare. In machine code, it is very difficult to do analysis to have
more information than just the decompiled text. We add some
information while simulating because we know for example the address
of specific trampolines, therefore we can print the name of the
trampoline when we see that its address is called. Again, sometimes we
also have to debug in gdb. In this case, we disassemble the machine
code and compare it to the one from in-image compilation, so both
printed strings have to be similar (similar text, same chariot returns).
Another example is the complexity of the Pharo tools:
While developing the VM, I have sometimes a VM partially working or
with some plugins not working. In the Squeak image, I can open a
workspace on top of this half-working VM and run do-its to see what is
working and what is not. In the Pharo image, I can't do anything. You
can't open the workspace without opening more advanced tools. I tried
to open the Playground, but the first time there was a bug with Traits
(Playground use Traits somehow and they were not working due to the
new bytecode set not being finished), when that first bug was fixed I
could not open it because it crashed simply the VM (I believe it tried
to access an external file such as playground-cache). Currently, the
Pharo team is trying to build a set of basic tools that have few
dependencies to debug a partially working system (that I think you
will use to debug glamour while editing it, because you cannot use the
glamour inspector if glamour is not working). That would solve this
issue.
But in no way this point is something that I can do alone to be able
to develop the VM in Pharo. This has to be a community effort. And I
am saying that because I can't be blamed not to work on the VM in
Pharo if to do so I need to spend many months changing Pharo.
An example that I believe is a problem in term of the community is the
following:
I added with Eliot the support for the new bytecode set. Currently,
the Squeak image works with the new bytecode set but not the Pharo
image. This is because only the Traits are broken, but this is
something I could hardly figure out in the Pharo image because nothing
is working as the GT tools use Traits. In Squeak I believe there are
very few users of Traits so everything worked, and the test suite can
reveal that the Traits are broken easily.
Currently, the VM process to me is to first make new features work in
Squeak, because it is simpler, and then make it work with Pharo, which
is more complex. In the last section I discussed how Traits were a
problem while implementing the new bytecode set. So what is the long
term solution for this issue ?
- Will we have a bootstrap process that creates first a Trait-free
Kernel and then build the Pharo Kernel out of it ?
- Do we forbid people to use Traits in the Pharo Kernel and does that
make sense to have Traits in Pharo in this case ?
- If we don't do anything, maybe the Traits are only a slight
difference with low impact in most cases and it's fine. But maybe
there are many small aspects like Traits, such as the Slots the way
they were used in GT recently (I don't blame GT or anything, it was
just using features in the system that created issues for me), and
maybe we reached a point where the complexity between the Pharo kernel
and the Squeak kernel is big enough so that a VM developer will first
make Squeak works when introducing new features and then deals with
the complexity of Pharo ?
So, what do we do ? I don't see any simple solution for this issue.
And I believe there are people around that see as the only solution
for this issue not to have the Pharo VM development process in Pharo
because they will see it as a threat to what they want to do with Pharo.
Best Doru !
PS: I am still using the GTInspector with additional views on graphs
created with Roassal everyday and I still enjoy it.
PS2: I am on vacation currently because I was getting crazy looking at
machine code all day long, so I may not answer as quick as usually
during the next week.
Cheers,
Doru
On Sat, May 9, 2015 at 9:31 PM, Clément Bera
<bera.clem...@gmail.com <mailto:bera.clem...@gmail.com>> wrote:
2015-05-09 20:25 GMT+02:00 stepharo <steph...@free.fr
<mailto:steph...@free.fr>>:
Le 9/5/15 20:16, Clément Bera a écrit :
This whole conversation here shows very well the point
that I tried to explain to Stef last week. I'm sorry if
the mail is a bit long but I think this discussion has to
be done.
My whole Smalltalk development life, I have used Pharo
and was happy with it. Now I am also working in Cog's JIT
compiler and for this specific project, I am working with
Squeak. I don't work with Squeak because I don't like
Pharo, I told you before, I have worked with Pharo on all
my project before, enjoyed it and if it was possible I
would use Pharo. I work with Squeak because the VM
development tool and development process simply does
*not* work in Pharo. This is not only because of VM tools
working with the old Morphic not working anymore in Pharo
or details like that, it is also due to deeper changes in
Pharo.
Stef believes it is important that Pharo is able to host
development for its own VM. Therefore, I discussed with
him and Esteban about a first list of points that are
necessary for Pharo to support its VM development in
Pharo, which includes this Transcript behavior.
As of today, and I am honest here, I believe that what is
required for Pharo to support the development process of
its VM includes points which goes in the opposite
direction than a few points in the Pharo roadmap, that
people in the Pharo community will see as a regression,
as "an intrusion from the Squeak philosophy into Pharo",
or as forbidding the integration of features that breaks
the VM development process. Therefore, I believe the
Pharo community would disapprove to make such changes and
I highly doubt that it is possible to have the
development process of the
Pharo VM in Pharo.
I was thinking that only a few points would be a problem
such as the increasing memory footprint of the Pharo
image that is going to get worse with the sources that
will be included in the image in the future, whereas a VM
developer needs a small image (See previous threads in
this mailing list where Hilaire complains about that for
example).
clement can I ask a simple question?
why did I ask guille to work on minikernels and bootstrap
for his phd instead on a topic where we can publish?
- choice A: lack of idea
- choice B: ....
I have already stated that you believe that it is important
that Pharo is able to host development for its own VM.
I am not against what you did and I am very excited with
Guille's work.
Pharo is community-driven, so I am not asking the question to
you only, but to the community.
However, I didn't think that even simple points like the
Transcript behavior discussed here, which looks like to
me as a regression and is required for VM development,
would be seen as an improvement by a non negligible part
of the community.
In this mailing-list, the whole Pharo community is
present and can see this discussion. So the open
questions are:
*Do you want to have the development of the Pharo VM in
Pharo, or do you want the development of the Pharo VM to
remain in Squeak ?*
*Do you think a system that is not good enough to handle
its own VM development is a good system ?*
I am not willing to go against the will of the community
because I enjoy community-driven softwares. If the answer
is that Pharo should be able to support its own VM
development then as I started I will help Esteban and
Stef to improve Pharo so that it can support its own VM
development. Now, if the answer is that the development
of the Pharo VM should remain in Squeak, I will continue
developing the VM in Squeak.
You are the Pharo community, you are the ones that make
Pharo alive and kicking, so you tell me what you think we
should do.
Clement
2015-05-09 18:23 GMT+02:00 Eliot Miranda
<eliot.mira...@gmail.com <mailto:eliot.mira...@gmail.com>>:
Hi Ben,
On May 9, 2015, at 7:41 AM, Ben Coman
<b...@openinworld.com <mailto:b...@openinworld.com>> wrote:
On Sat, May 9, 2015 at 10:09 PM, Ben Coman
<b...@openinworld.com <mailto:b...@openinworld.com>>
wrote:
From my limited experience bug hunting, calling
#changed: from a thread other than the UI thread
is a source of evil. There are too many
assumptions throughout the system that the UI is
single threaded. Can anyone advise me that is
not a proper belief?
Then that implies that a Transcript
implementation where #nextPut: direct calls
#changed:
is not appropriate for use with multi-threaded
applications. In Pharo, #changed: is only called
from #stepGlobal, which is called from
doOneCycle:. (This came about as a last minute
bug fix before Pharo 3 release and maybe could
use some cleanup.
Separating the UI from Transcript into its own
viewer might be a good idea, but actually it
would not solve Stef's case since his code would
still be running in the UI thread -- unless the
viewer ran in another thread, which would have
its own complexities.
I think the point about efficiency is
significant. The following example...
Time millisecondsToRun: [ 1000 timesRepeat:
[ Transcript show: 'x' ] ]
on Squeak 4.5 --> 12749ms
on Pharo 50029 --> 2ms
As a point of comparison, on VW 8.0 --> 43817ms
and so you might guess, VW 8.0 outputs each 'x'
immediately.
cheers -ben
Way to go, Squeak! Actually this is disappointing.
I'm rather frustrated with Squeak's slow transcript,
and was hoping that VW would demonstrate it could be
faster. Looking at the Squeak implementation I only
see an obvious 30% or so improvement via tuning.
Looks like good performance will take more work :-/
Eliot (phone)
--
www.tudorgirba.com <http://www.tudorgirba.com>
"Every thing has its own flow"
