Re: [drlvm][design] class unloading: Secondary root set

[Robin Garner]

Salikh Zakirov wrote:

Salikh Zakirov wrote:

I think you have missed one point: after retracing from secondary root
set once,
more classloaders may be found reachable, so this step needs to be
repeated until
convergence (to obtain the closure of reachability with additional
links Object->Class,
served through vtable marks).


Robin Garner wrote:

My proposal doesn't require steps (2) although VM->ClassLaoder
references are weak, and (5), because the trace from the vtable roots is
no different fromthe standard GC trace.


Okay, It looks like you found a way to avoid repetitive retracing from added 
roots,
and doing it in one step. If this is the case, I see how one can do
with just enumerating vtables, and without "unload list".

However, I do not understand how can correctness and completeness can be 
achieved
with just one retracing. Java allows for arbitrary implementation of 
user-defined
class loaders, including chained (think of java application server loaded as an
application to another application server). 


Because: By tracing from the live vtables, you are using the complete 
transitive closure mechanism of the GC.  Once a classloader becomes 
reachable, its pointers will be enumerated, all the classes that belong 
to it will be traced, and anything reachable from them will be live, 
including other classloaders.


Or am I missing something ?

Wait, I suppose if j.l.ClassLoaders can point to ordinary heap objects 
then they in turn could revive vtables which could revive classloaders - 
yes, you might need to iterate until no new vtables are revived.



You could alternately say that I'm simply refining your approach.  Yes,
they are structurally very similar - if you agree with my refinements,
feel free to merge them.


I will gladly merge your ideas as soon as I understand them,
but unfortunately I cannot see how can algorithm be correct without
transitive classloader revival.

It looks to me like one-step approach at deciding whether on unloading a classloader 
can produce incorrect results in the case of multiple chained classloaders.

For example, the test below will unload two of the classloaders incorrectly
at the first System.gc(). Note, that the test works correctly on Sun Hotspot:

$ java -showversion -verbose:gc UnloadTwice
java version "1.5.0_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-b05)
Java HotSpot(TM) Client VM (build 1.5.0_05-b05, mixed mode)
loading UnloadTwice.class
loading UnloadTwice.class
loading UnloadTwice.class
[Full GC 284K->131K(1984K), 0.0100915 secs]  <<<<<<<< it does not unload any 
classloader on the first GC
[Full GC[Unloading class UnloadTwice]<<<<<<<< it unloads all 3 
classloaders at the second GC
[Unloading class UnloadTwice]
[Unloading class UnloadTwice]
 131K->131K(1984K), 0.0092772 secs]
ok



import java.io.*;
import java.lang.reflect.*;

public class UnloadTwice extends ClassLoader {

static Object o;

public static void init() {
try {
UnloadTwice cl1 = new UnloadTwice();
ClassLoader scl = ClassLoader.getSystemClassLoader();
cl1.setParent(scl);
Class c1 = cl1.loadClass("UnloadTwice");
Method sp1 = c1.getMethod("setParent", ClassLoader.class);
ClassLoader cl2 = (ClassLoader)c1.newInstance();
sp1.invoke(cl2, new Object[] { scl });
Class c2 = cl2.loadClass("UnloadTwice");
Method sp2 = c2.getMethod("setParent", ClassLoader.class);
ClassLoader cl3 = (ClassLoader)c2.newInstance();
sp2.invoke(cl3, new Object[] { scl });
Class c3 = cl3.loadClass("UnloadTwice");
o = c3.newInstance();
} catch (Exception e) {
e.printStackTrace();
}
}

public static void main(String[] args) {
init();
System.gc(); // we still have third-level-chained object o live
o = null;
System.gc(); // now o is gone, class unloading should happen
System.out.println("ok");
}

ClassLoader parent;

public void setParent(ClassLoader parent) {
this.parent = parent;
}

public Class loadClass(String name) {
try {
if (!"UnloadTwice".equals(name)) return parent.loadClass(name);
System.out.println("loading " + name + ".class");
InputStream in = parent.getResourceAsStream(name + ".class");
byte bytes[] = new byte[in.available()];
in.read(bytes);
return defineClass(bytes, 0, bytes.length);
} catch (Exception e) { throw new RuntimeException(e); }
}
}





--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [design] class unloading: Secondary root set

[Robin Garner]

Salikh Zakirov wrote:

Robin,

I have read your description of "secondary root set" approach, and found it the 
same as
"finalization-like" approach discussed by Etienne and me earlier.

I think you have missed one point: after retracing from secondary root set once,
more classloaders may be found reachable, so this step needs to be repeated 
until
convergence (to obtain the closure of reachability with additional links 
Object->Class,
served through vtable marks).

Can you reread two proposals and merge the text into one section?
I can do it, but want first to make sure we have common understanding.



My proposal doesn't require steps (2) although VM->ClassLaoder 
references are weak, and (5), because the trace from the vtable roots is 
no different fromthe standard GC trace.


You could alternately say that I'm simply refining your approach.  Yes, 
they are structurally very similar - if you agree with my refinements, 
feel free to merge them.


--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

[drlvm] Re: [design] Class unloading: false sharing of vtable marks

[Robin Garner]

.


Calculating the address of the mark bit is a simple mask and shift 
operation on the address of the mark bit (pure register/ALU, v fast, 
overlapped with memory fetches).


hope this helps,
Robin


(*
This is a follow-up to design proposals at 
http://wiki.apache.org/harmony/ClassUnloading


I am starting new discussion because mailing list is a better means for 
discussion
than Wiki. After we come to conclusion, I will log it to the wiki page.
*)





--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Ivan Volosyuk wrote:

On 11/9/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:

Ivan Volosyuk wrote:
> We will get rid of false sharing. That's true. But it still be quite
> expensive to write those '1' values, because of ping-ponging of the
> cache line between processors. I see only one solution to this: use
> separate mark bits in vtable per GC thread which should reside in
> different cache lines and different from that word containing gcmap
> pointer.

The only thing that a GC thread does is write "1" in this slot; it never
writes "0".  So, it is not very important in what order (or even "when")
this word is finally commited to main memory.  As long as there is some
barrier before the "end of epoch collection" insuring that all
processors cache write buffers are commited to memory before tracing
vtables (or gc maps).

You don't need memory coherency on write-without-read. :-)


I don't speak about memory coherency, I speak about bus load with
useless memory traffic between processors and poor CPU cache usage.

Surely this wouldn't happen in a sufficiently weak memory model ?  Lets 
just not support x64 :-)


But I think this false sharing may be what kills this particular idea.
The next cheapest option should be to use a side array of bytes - as 
long as calculating the address of the mark byte can be done without any 
loads or register spills, it should still be cheaper than a full 
test-and-mark operation on the vtable.  I guess there are cache policies 
where this may still be slow on an SMP machine.


Side metadata is easiest to do when objects are in a specific space, and 
has coarse alignment.  Any ideas what the typical size of a DRLVM vtable 
is ?  Would 256 bytes be an excessive alignment boundary ?


I'll try it out in the next day or so.  Unfortunately I don't have 
access to anything with more parallelism than a Pentium D, so it's not 
likely to be conclusive.


--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm] vote on class unloading design (was Class unloading support - tested one approach)

[Robin Garner]

Etienne Gagnon wrote:

Aleksey Ignatenko wrote:

So the list is:
[...]
2. per heap space/generation boolean marker on the classloader instance(
Etienne )
3. class reachability marker byte in vtable (Robin )
[...]


Unless Robin disagrees(?), I would say that 2 and 3 have evolved into a
single merged proposal that comes with various minor variations for
different collection needs: simple "stop the world" (stw), generational
stw, concurrent, etc.  You could call it the "epoch-based vtable
marking" approach.

So, that bring us back to 3.

Robin: Do you agree?  :-)

Etienne


I agree.

--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Geir Magnusson Jr. wrote:



Robin Garner wrote:

Geir Magnusson Jr. wrote:
Nice - we should put this on the website (maybe same place where we 
point to JAPI results).


Are you tracking performance #'s over time?  You are just tracking 
regression in terms of how much runs, rather than how fast?


geir


Right now it's just regression failures, but I'm working on graphs of 
performance over time, for the non-commercial VMs at least.


Do 'em all!  We need to track ourselves to Sun et al, because that's 
what we're trying to achieve :)


Well ... I thought there were legal issues wrt releasing performance 
numbers of some of these VMs.




This is a temporary home, so don't link to it too permanently :)


ok.  We'll use the ...


:-P


geir



cheers



Robin Garner wrote:



Weldon Washburn wrote:

Fantastic!

By the way, the http:// link below did not work for me.


Doh!

  http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/



On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


I've just finished adding drlvm to the nightly DaCapo benchmark
regression tests.  The results are available here:

http://cs/people/Robin.Garner/dacapo/regression/

JikesRVM and DRLVM/Harmony classlib are built from svn checkouts 
taken

when the nightly run kicks off (00:35 Australian Eastern time).

I'm currently working on adding performance stats for DRLVM and
JikesRVM, so some suggestions about the best command line 
parameters to

use would be appreciated.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University














--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [crypto] SHA-1 not implemented?

[Robin Garner]

Stefano Mazzocchi wrote:

from Robin's latest runs
 
http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/results-20061110/DRLVM/eclipse.small.log

there are a bunch of log messages that indicate that harmony doesn't
implement SHA-1.

Is that true?



It can't be true, because _all_ the DaCapo benchmarks rely on SHA-1 for 
validation.  I raised JIRA Harmony-2135 on this issue.  Looks like after 
eclipse has run, drlvm forgets how to access the SHA-1 algorithm :(


--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm] vote on class unloading design (was Class unloading support - tested one approach)

[Robin Garner]

Weldon Washburn wrote:

It looks like I called for a vote too soon.  The continuing discussion on
class unloading design is uncovering many important issues.  This is
excellent as it is much better to deal with design issues at this stage
rather than during implementation.

I propose the following:

1)
Cancel the current vote on design.

2)
Someone put together a complete class unloading design based on
Etienne/Robin's approach.  Include pseudo code and post to harmony-dev.

3)
We call for a new vote once the comments on the documented design indicate
it is ready.



Could I also suggest that maybe the mailing list isn't the most ideal 
place to keep draft design documents ?  Isn't there a wiki or some other 
web page where it could be kept and referred to ?


--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Etienne Gagnon wrote:

OK.  My latest proposal (a few messages ago) was assuming that the
nursery was empty when the "end of epoch collection" is launched.

If it is not, you can do 2 things:

a) do a minor collection to empty it, or

b) i  - use a finalization-like list of references to class loader
objects
   ii - launch gc, which might mark a previously unmarked vtable
   iii- do a finalization-like rescuing for resuscitated class loaders

"b)" should really have a minimal performance impact.  As for its
"apparent complexity", I would say that this is a non-issue; similar
code must already exist in drlvm for implementing finalization.


Just for clarification: "b)" implies a combined "nursery + mature space"
collection.

Actually, for the mature space part, you could get away with a smaller
collection if you premature all class loaders and classes to a specific
mature space area; the you only need to collect that space (in addition
to the nursery).

Etienne



This sounds rather 'stop the world' - while the barrier is more 
complicated I think it scales to concurrent collectors.


Also, don't forget an instance of a class in the nursery can pass a 
reference to its classloader to a mature-space object under suitably 
bizarre circumstances.  I guess you could have a write barrier on the 
class metadata space ...


... an XOR barrier could actually be an interesting solution ... but I'm 
sure it won't be necessary.


--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Alexey Varlamov wrote:

Robin,

1) it must be debug build of drlvm;
2) syntax is -Xstats:[1|2|3] , the number denotes detalization level.

Actually this options gives various *statistics* for run, rather than
performance data or profile. E.g. number of particular byte ops
performed, exceptions thrown/caugth, most frequently allocated
classes, usage of internal memory pools, etc.

I'm not sure this is what you requested - do you need to tune the VM
for best performance or collect additional info during run or smth
else?


Actually I'm just interested in wall clock time for the purpose of 
direct performance comparisons, but I always have a use for performance 
statistics from JVMs.  Some of the numbers you list are tricky to get 
from my measurement infrastructure :)


thanks!



2006/11/9, Robin Garner <[EMAIL PROTECTED]>:

Hmmm ...

$ $DRLVM_HOME/jre/bin/java -Xbootclasspath/p:antlr.jar -XcleanupOnExit
-Xstats -jar $HOME/dacapo-2006-10.jar antlr
Unknown option -Xstats
Use java -help to get help on command line options

$ $DRLVM_HOME/jre/bin/java -version
Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
Foundation or its licensors, as applicable.
java version "1.5.0"
pre-alpha : not complete or compatible
svn = r472802, (Nov  9 2006), Linux/ia32/gcc 4.0.3, release build
http://incubator.apache.org/harmony

Anything else I need to add ? :-)

Evgueni Brevnov wrote:
> Unfortunately, YES. It's a known problem. You need to specify
> -XcleanupOnExit as well :-) I'm working in it currently will
> provide a fix soon...
>
> Thanks
> Evgueni
>
> On 11/9/06, Robin Garner <[EMAIL PROTECTED]> wrote:
>> Evgueni Brevnov wrote:
>> > On 11/9/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:
>> >> The logs for some benchmarks does not have error messages.
>> >> Is this possible to fix?
>> >>
>> >> SY, Alexey
>> >>
>> >> 2006/11/9, Robin Garner <[EMAIL PROTECTED]>:
>> >> > I've just finished adding drlvm to the nightly DaCapo benchmark
>> >> > regression tests.  The results are available here:
>> >> >
>> >> > http://cs/people/Robin.Garner/dacapo/regression/
>> >> >
>> >> > JikesRVM and DRLVM/Harmony classlib are built from svn checkouts
>> taken
>> >> > when the nightly run kicks off (00:35 Australian Eastern time).
>> >> >
>> >> > I'm currently working on adding performance stats for DRLVM and
>> >> > JikesRVM, so some suggestions about the best command line
>> parameters to
>> >> > use would be appreciated.
>> >
>> > DRLVM accepts -Xstats parameter for that purpose.
>>
>> -Xstats doesn't work for me - is there something I need to do
>> specifically to make it work ?
>>
>> > Thanks
>> >
>> >> >
>> >> > cheers
>> >> >
>> >> > --
>> >> > Robin Garner
>> >> > Dept. of Computer Science
>> >> > Australian National University
>> >> >
>> >>
>>
>>
>> --
>> Robin Garner
>> Dept. of Computer Science
>> Australian National University
>>


--
Robin Garner
Dept. of Computer Science
Australian National University




--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Alexey Varlamov wrote:

2006/11/9, Robin Garner <[EMAIL PROTECTED]>:

Etienne Gagnon wrote:
> Alexey Varlamov wrote:
>> Sorry if it was already discussed, but I believe this approach also
>> requires marking vtable bit/byte on each object allocation, unitl the
>> "unloading" GC pass is strictly stop-the-world full-heap collection.
>> Robin, did you include this particular overhead too in your
>> measurements?

I didn't include it - having established that it's cheap during GC where
memory bandwidth is at a premium, I kind of took this for granted.

> My proposal already argued that vtable bit/byte/word marking is
> unnecessary for "nursery allocations".  You only need to mark the 
vtable

> of objects that survive collection and pretenured objects.


I may have missed it, but I only recall you argued that we just need
to collect mature space for the *final unloading* as CL and classes
are unlikely to die young, which I agree. But chances that a live
object of a candidate class appeared in the nursery are higher.
Otherwise I just do not grok how this algorithm can be proven for 
correctness.


There is definitely some kind of barrier required here.  If no 
references to classes belonging to a c/l exist, but references to one of 
the j.l.classloaders exist, classloader may get marked for collection. 
Objects get created (via reflection, in nursery), references to c/l are 
dropped, classloader unloads.


I believe a barrier in one or more of the reflective methods used to 
create objects from j.l.class/j.l.c/loader references is probably necessary.


Weak references can only be collected at the end of a reachability epoch 
in any case, so I think there may be some stronger guarantees that we 
can use, but I'm too sleepy to thing of them right now :)



And this is a persuasive argument.  But I can probably find time to
measure it tomorrow if you aren't convinced.


That would be very kindly, thank you.

--
Alexey


--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Hmmm ...

$ $DRLVM_HOME/jre/bin/java -Xbootclasspath/p:antlr.jar -XcleanupOnExit 
-Xstats -jar $HOME/dacapo-2006-10.jar antlr

Unknown option -Xstats
Use java -help to get help on command line options

$ $DRLVM_HOME/jre/bin/java -version
Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software 
Foundation or its licensors, as applicable.

java version "1.5.0"
pre-alpha : not complete or compatible
svn = r472802, (Nov  9 2006), Linux/ia32/gcc 4.0.3, release build
http://incubator.apache.org/harmony

Anything else I need to add ? :-)

Evgueni Brevnov wrote:

Unfortunately, YES. It's a known problem. You need to specify
-XcleanupOnExit as well :-) I'm working in it currently will
provide a fix soon...

Thanks
Evgueni

On 11/9/06, Robin Garner <[EMAIL PROTECTED]> wrote:

Evgueni Brevnov wrote:
> On 11/9/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:
>> The logs for some benchmarks does not have error messages.
>> Is this possible to fix?
>>
>> SY, Alexey
>>
>> 2006/11/9, Robin Garner <[EMAIL PROTECTED]>:
>> > I've just finished adding drlvm to the nightly DaCapo benchmark
>> > regression tests.  The results are available here:
>> >
>> > http://cs/people/Robin.Garner/dacapo/regression/
>> >
>> > JikesRVM and DRLVM/Harmony classlib are built from svn checkouts 
taken

>> > when the nightly run kicks off (00:35 Australian Eastern time).
>> >
>> > I'm currently working on adding performance stats for DRLVM and
>> > JikesRVM, so some suggestions about the best command line 
parameters to

>> > use would be appreciated.
>
> DRLVM accepts -Xstats parameter for that purpose.

-Xstats doesn't work for me - is there something I need to do
specifically to make it work ?

> Thanks
>
>> >
>> > cheers
>> >
>> > --
>> > Robin Garner
>> > Dept. of Computer Science
>> > Australian National University
>> >
>>


--
Robin Garner
Dept. of Computer Science
Australian National University




--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Etienne Gagnon wrote:

Alexey Varlamov wrote:

Sorry if it was already discussed, but I believe this approach also
requires marking vtable bit/byte on each object allocation, unitl the
"unloading" GC pass is strictly stop-the-world full-heap collection.
Robin, did you include this particular overhead too in your
measurements?


I didn't include it - having established that it's cheap during GC where 
memory bandwidth is at a premium, I kind of took this for granted.



My proposal already argued that vtable bit/byte/word marking is
unnecessary for "nursery allocations".  You only need to mark the vtable
of objects that survive collection and pretenured objects.


And this is a persuasive argument.  But I can probably find time to 
measure it tomorrow if you aren't convinced.


--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][sablevm] Desing of Class Unloading Support

[Robin Garner]

Geir Magnusson Jr. wrote:



Weldon Washburn wrote:



On 11/8/06, *Geir Magnusson Jr.* <[EMAIL PROTECTED] 
<mailto:[EMAIL PROTECTED]>> wrote:




Weldon Washburn wrote:
 > On 11/7/06, Ivan Volosyuk < [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
 >>
 >> On 07 Nov 2006 14:35:55 +0600, Egor Pasko <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
 >> > > I already have one idea how to benefit from movable vtables.
 >
 >
 > There would have to be a very compelling argument for making 
vtables

 > movable.  Like a business workload that Harmony needs to run
within the
 > next
 > 12 months.

How would a business workload need this directly?
 
That's the point.  I can't figure out any compelling story for moving 
vtables.  As far as I can tell, its over-engineering.   I would love 
to be proven wrong.


But isn't this simply an implementation detail of something that is 
important, namely the class unloading?


geir


While it did come up as an issue in the class-unloading talks I think 
most of us believe it to be orthogonal.


cheers

--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Geir Magnusson Jr. wrote:
Nice - we should put this on the website (maybe same place where we 
point to JAPI results).


Are you tracking performance #'s over time?  You are just tracking 
regression in terms of how much runs, rather than how fast?


geir


Right now it's just regression failures, but I'm working on graphs of 
performance over time, for the non-commercial VMs at least.


This is a temporary home, so don't link to it too permanently :)

cheers



Robin Garner wrote:



Weldon Washburn wrote:

Fantastic!

By the way, the http:// link below did not work for me.


Doh!

  http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/



On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


I've just finished adding drlvm to the nightly DaCapo benchmark
regression tests.  The results are available here:

http://cs/people/Robin.Garner/dacapo/regression/

JikesRVM and DRLVM/Harmony classlib are built from svn checkouts taken
when the nightly run kicks off (00:35 Australian Eastern time).

I'm currently working on adding performance stats for DRLVM and
JikesRVM, so some suggestions about the best command line parameters to
use would be appreciated.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University











--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Evgueni Brevnov wrote:

On 11/9/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:

The logs for some benchmarks does not have error messages.
Is this possible to fix?

SY, Alexey

2006/11/9, Robin Garner <[EMAIL PROTECTED]>:
> I've just finished adding drlvm to the nightly DaCapo benchmark
> regression tests.  The results are available here:
>
> http://cs/people/Robin.Garner/dacapo/regression/
>
> JikesRVM and DRLVM/Harmony classlib are built from svn checkouts taken
> when the nightly run kicks off (00:35 Australian Eastern time).
>
> I'm currently working on adding performance stats for DRLVM and
> JikesRVM, so some suggestions about the best command line parameters to
> use would be appreciated.


DRLVM accepts -Xstats parameter for that purpose.


-Xstats doesn't work for me - is there something I need to do 
specifically to make it work ?



Thanks


>
> cheers
>
> --
> Robin Garner
> Dept. of Computer Science
> Australian National University
>




--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Evgueni Brevnov wrote:

On 11/9/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:

The logs for some benchmarks does not have error messages.
Is this possible to fix?

SY, Alexey

2006/11/9, Robin Garner <[EMAIL PROTECTED]>:
> I've just finished adding drlvm to the nightly DaCapo benchmark
> regression tests.  The results are available here:
>
> http://cs/people/Robin.Garner/dacapo/regression/
>
> JikesRVM and DRLVM/Harmony classlib are built from svn checkouts taken
> when the nightly run kicks off (00:35 Australian Eastern time).
>
> I'm currently working on adding performance stats for DRLVM and
> JikesRVM, so some suggestions about the best command line parameters to
> use would be appreciated.


DRLVM accepts -Xstats parameter for that purpose.


OK, thanks for that.

I was more thinking about making it go fast :)  Or is the default 
designed to be as fast as possible ?


cheers

--
Robin Garner
Dept. of Computer Science
Australian National University

[classlib] DaCapo chart benchmark

[Robin Garner]

While the DaCapo 'chart' benchmark runs successfully, it differs from 
the production VMs (on Linux) in that it requires a valid X-windows 
display.  Gnu Classpath also has this problem.


From conversations with the JFreeChart maintainer, we know that 
JFreeChart relies heavily on Java2D and parts of AWT.  My guess is that 
the harmony and classpath libraries are more eager than the production 
libraries in initializing the X windows display.


For the record, the 'chart' benchmark produces Pdf output from a graph 
specification, so there's no reasonable expectation that it would 
require a display.


Not what I'd call a problem, but someone might like to add it to their list.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Alexey Petrenko wrote:

Robin,

I was frightened of your  message subject... It looks like you've
found some serious regressions in classlib and drlvm :)


:)  Well, actually only 7 of the 11 benchmarks run - I guess it's only a 
regression if they *ever* ran :)




The results are available here:
http://cs/people/Robin.Garner/dacapo/regression/

It looks like my notebook does not know where to find host named cs :(



See other e-mail :-/


--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

Weldon Washburn wrote:

Fantastic!

By the way, the http:// link below did not work for me.


Doh!

  http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/



On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


I've just finished adding drlvm to the nightly DaCapo benchmark
regression tests.  The results are available here:

http://cs/people/Robin.Garner/dacapo/regression/

JikesRVM and DRLVM/Harmony classlib are built from svn checkouts taken
when the nightly run kicks off (00:35 Australian Eastern time).

I'm currently working on adding performance stats for DRLVM and
JikesRVM, so some suggestions about the best command line parameters to
use would be appreciated.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University








--
Robin Garner
Dept. of Computer Science
Australian National University

[drlvm][classlib] DaCapo benchmark regressions

[Robin Garner]

I've just finished adding drlvm to the nightly DaCapo benchmark 
regression tests.  The results are available here:


http://cs/people/Robin.Garner/dacapo/regression/

JikesRVM and DRLVM/Harmony classlib are built from svn checkouts taken 
when the nightly run kicks off (00:35 Australian Eastern time).


I'm currently working on adding performance stats for DRLVM and 
JikesRVM, so some suggestions about the best command line parameters to 
use would be appreciated.


cheers

--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Etienne Gagnon wrote:

3- Why would it be so hard to add an unconditional write operation
during collection (e.g. during copying or marking of an object) in
drlvm?  A detailed technical explanation is welcome. :-)


I actually believe that this should be implementable in a GC-neutral 
way, whether vtables are objects or not.  The GC will at some point ask 
the VM for the GC map of the object it is about to scan.  At this point 
the VM can write the mark of the vtable.


I guess I'm making an assumption about the GC -> VM interface here, but 
if it doesn't exist, it should :)



  So far, this latest point (3-) seems the sole argument in favor of
  using the "object-vtables" approach.  Wouldn't the right fix, if
  it's currently really impossible to implement an unconditional
  write, be to extend the drlvm GC interface?  Isn't this a design
  deficiency in the GC interface?  No other argument, so far, seems
  to be in favor of the object vtable approach, unless I missed some.


As for Robin's attempt to deal with weak/hard reference to the class
loader object using a Reference Queue, I am not yet convinced of the
correctness of the approach...  [off the top of my head: potential
problem with static synchronized methods].  And, this would be probably
more work intensive (so, less efficient) than my proposal in 1-[*]
above.  It might also be tricky to identify all possible situations such
as Object.getClass() where some special code is needed to deal with a
problem situation.  I prefer clean, scope-limited code for dealing with
class unloading.  It's easier, that way, to activate or deactivate class
loading [dynamically!].


Glad to see I'm not the only one :)  It really was just off the top of 
my head.



In summary, I would like to be convinced of the "completeness" and of
the "correctness" of all competing approaches.  I personally am, so far,
in favor of Robin's unconditional vtable bit/byte/word write idea, along
with an "adapted" version of my proposal for dealing with class loader
death (such as proposed in 1-[*] above).

Also, if somebody was able to find a "correctness" deficiency in my
proposal, then please let us know, so that we make sure this deficiency
is eliminated from all competing proposals.

By the way, what are the currently competing proposals?
1- object vtables
2- Robin/Gagnon proposal  (still finishing up some details ;-)
3- Is there a 3rd?

Which ones have existing implementations?  How "correct/complete" are
they?  Do we have access to some "human readable" (i.e. non-code) full
description of the algorithm?


And what is their performance hit ?


Etienne

Robin wrote:

On Thu, 2006-11-09 at 02:01 +0300, Ivan Volosyuk wrote:


Robin,

thank you for detailed description of the algorithm. IMHO, this was
the most complicated place of the whole story: how to have a weak
reference to classloader and still be able to get it alive again. This
shouldn't be performance critical part and is quite doable. I
absolutely agree with your estimations about tracing extra reference
per object. The approach you propose is more efficient and quite
elegant.
--
Ivan


Thanks :)



On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Robin Garner wrote:


Aleksey Ignatenko wrote:


Robin.



OK, well how about keeping a weak reference to the >j.l.ClassLoader
object instead of a strong one.  When the reference >becomes (strong)ly
unreachable, invoke the class-unloading phase.


If you have weak reference to j.l.Classloader - GC will collect it
(with all
appropriate jlClasses) as soon as there are no references to
j.l.Classloaderand appropriate classes. But there is possible
situation when there are some
live objects of that classes and no references to jlClassloader and
jlClasses. This will lead to unpredictable consequences (crash, etc).



I want to remind that there 3 mandatory conditions of class unloading:

1. j.l.Classloader instance is unreachable.

2. Appropriate j.l.Class instances are unreachable.

3. No object of any class loaded by appropriate class loader exists.

Let me repeat.  I offer an efficient solution to (3).  I don't purport
to have a solution to (1) and (2).

Let me just add:  This is because I don't think (1) or (2) are
particularly difficult from a performance point of view, although I'm
happy to accept that there may still be some subtle engineering challenges.

Now this is just off the top of my head, but what about this for a design:
- A j.l.ClassLoader maintains a collection of each of the classes it has
loaded
- A j.l.Class contains a pointer to its j.l.ClassLoader
- A j.l.Class maintains a collection of its vtable(s) (or a pointer if 1:1).
The point of this is that a class loader and its classes are a 'self
sustaining' data structure - if one element in it is reachable the whole
thing is reachable.

The VM m

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Etienne Gagnon wrote:

I was making it more complex than it needs...

Here's an improvement...

1- During normal operation, the VM keeps hard references to all class
loader instances.  [This prevents any premature class loader death].

2- At the start of an epoch (or just before), all vtable bits (or byte
or word) are cleared. [From now on, I will use the "bit" terminology for
simplicity.  The bit may reside in an otherwise unused byte or even
word, for efficiency purpose].

3- The end of an epoch happens "no sooner" than when all generations /
heap parts have been collected at least once since the epoch start.
[One can cheat and visit objects of uncollected parts/generations to
mark their vtables].

4- An "old generation" collection is chosen as the end of an epoch.
This is "end of epoch collection".  [As class loaders/classes are likely
to have moved to older generations, there's no point trying to kill them
in young collections].


In fact classes and clasloaders would be perfect targets for pretenuring.


5- Just before starting the "end of epoch collection", all the
class-loader vtable lists are visited (and bits are cleared in prevision
of the next epoch).  All vm references to [candidate] class loaders with
no surviving objects (nor active methods) (e.g. no vtable bit set) are
made "weak".

6- The "end of epoch collection" is launched.

7- There's actually no need for "rescuing" class loaders.  The vm
reference to any surviving [candidate] class loader is made hard again.
   Interesting fact: other candidate class loaders cannot have any
instance (nor any active method) as GC doesn't create instances nor
method calls.  So, there's no need for a rescuing dance!  The list of
dying class loaders can be used for freeing related native resources.

IMO: simple, clean, efficient...


It has the downside of being inherently 'stop the world', though.  I 
don't see this as being a big disadvantage, because it shouldn't be hard 
(compared to the work of building a concurrent collector in the first 
place) to extend to a concurrent class-unloader.



Etienne

Etienne Gagnon wrote:

 If it does, then can somebody explain to me what's wrong with
 my proposal of setting, in normal operation, a "hard" reference to
 class loader objects, and then temporarily using weak, rescuable
 reference to enable class loader collection?  I don't see a performance
 hog there.  Rescuing a few class loaders (if any) and their related
 classes once per epoch shouldn't cost much!  I have yet to see a
 convincing explanation of how continuous collection of "object-vtables"
 would be more efficient...

 Really, even with Robin's proposal, this would work.  If a class loader
 gets into an unloadable state, then most likely, the class loader and
 all classes it has loaded will have migrated to an old generation.  So,
 as long as we set then end of a class unloading epoch at an old
 generation collection, then we can simply "weaken" the class loader
 reference during collection (only when the bit of all related vtables
 are unset), then apply the finalization-like rescue dance to class
 loaders.

 [*]This wouldn't affect any other operation, during other GC cycles, as
 Robin's unconditional bit/byte/word vtable write only serves to tell us
 whether a class loader has had living instances of its classes during
 the epoch.





--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Robin Garner wrote:

Aleksey Ignatenko wrote:

Robin.


OK, well how about keeping a weak reference to the >j.l.ClassLoader
object instead of a strong one.  When the reference >becomes (strong)ly
unreachable, invoke the class-unloading phase.



If you have weak reference to j.l.Classloader - GC will collect it 
(with all

appropriate jlClasses) as soon as there are no references to
j.l.Classloaderand appropriate classes. But there is possible
situation when there are some
live objects of that classes and no references to jlClassloader and
jlClasses. This will lead to unpredictable consequences (crash, etc).



I want to remind that there 3 mandatory conditions of class unloading:

1. j.l.Classloader instance is unreachable.

2. Appropriate j.l.Class instances are unreachable.

3. No object of any class loaded by appropriate class loader exists.


Let me repeat.  I offer an efficient solution to (3).  I don't purport 
to have a solution to (1) and (2).


Let me just add:  This is because I don't think (1) or (2) are 
particularly difficult from a performance point of view, although I'm 
happy to accept that there may still be some subtle engineering challenges.


Now this is just off the top of my head, but what about this for a design:
- A j.l.ClassLoader maintains a collection of each of the classes it has 
loaded

- A j.l.Class contains a pointer to its j.l.ClassLoader
- A j.l.Class maintains a collection of its vtable(s) (or a pointer if 1:1).
The point of this is that a class loader and its classes are a 'self 
sustaining' data structure - if one element in it is reachable the whole 
thing is reachable.


The VM maintains a weak reference to all its j.l.ClassLoader instances, 
and maintains a ReferenceQueue for weakly-reachable classloaders. 
ClassLoaders are placed on the ReferenceQueue if and only if they are 
unreachable from the heap (including via their j.l.Class objects).  Note 
this is an irreversible condition: objects that are unreachable can 
never become reachable again, except through very specific methods.


When it sweeps the ReferenceQueue for unreachable classloaders, the VM 
places the unreachable classloaders in a queue of classloaders that are 
candidates for unloading.  This queue is part of the root set of the VM. 
 A classloader in this queue is unreachable from the heap, and can be 
unloaded when there are no objects of any class it has loaded.


This is where my mechanism comes into play.

If an object executes getClass() then its classloader is removed from 
the unloadable classloader queue, its weak reference gets recreated  and 
we're back at the initial state.  My guess is that this is a pretty 
infrequent method call.


I think this stage of the algorithm is easy in performance terms - 
difficult in terms of proving correctness, but if you have an efficient 
reachability mechanism for classes I think the building blocks are 
there, and the subtleties are nothing that a talented engineer can't solve.



I'm not 100% sure what your counter-proposal is: I recall 2 approaches 
from the mailing list:

1) Each object has an additional word in its header that points back to
   its j.l.Class object, and we proceed from here.

Given that the mean object size is ~28 bytes, this proposal adds 14% to 
each object size.  This increases the frequency of GC by 14% and incurs 
a 14% slowdown.  Of course this is an oversimplification but a 14% 
slowdown is a pretty lousy starting point to argue from.


2) The existing pointer in the GC header is traced during GC time.

The average number of pointers per object (excluding the vtable) is 
between 1.5 and 2 for the majority of benchmarks I have looked at 
(footnote: if you know something different, drop me a line) (geometric 
mean 1.78 for {specJVM, pseudoJBB and DaCapo 20051009}).  Tracing one 
additional reference per object will therefore increase the cost of GC 
by ~60% on average.  Again oversimplification but indicative.  If we 
assume that GC accounts for 10% of runtime (more or less depending on 
heap size), this is a runtime overhead of 6%.


My proposal has been measured at ~1% overhead in GC time, or 0.1% in 
execution time (caveats as above).  If there is some complexity in 
establishing classloader reachability from this basis, I would assume it 
can easliy be absorbed.


Therefore I think my proposal, while not complete, can form the basis of 
an efficient complete system for class unloading.


(PS: I'd *love* to be proven wrong)

cheers,
Robin


Regards,
Robin




Aleksey.


On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Pavel Pervov wrote:
> Robin,
>
> The kind of model I had in mind was along the lines of:
>> - VM maintains a linked list (or other collection type) of the
currently
>> loaded classloaders, each of which in turn maintains the 
collection of
>> classes loaded by that type.  The sweep of classloaders goes 
somethin

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Aleksey Ignatenko wrote:

Robin.


OK, well how about keeping a weak reference to the >j.l.ClassLoader
object instead of a strong one.  When the reference >becomes (strong)ly
unreachable, invoke the class-unloading phase.



If you have weak reference to j.l.Classloader - GC will collect it (with 
all

appropriate jlClasses) as soon as there are no references to
j.l.Classloaderand appropriate classes. But there is possible
situation when there are some
live objects of that classes and no references to jlClassloader and
jlClasses. This will lead to unpredictable consequences (crash, etc).



I want to remind that there 3 mandatory conditions of class unloading:

1. j.l.Classloader instance is unreachable.

2. Appropriate j.l.Class instances are unreachable.

3. No object of any class loaded by appropriate class loader exists.


Let me repeat.  I offer an efficient solution to (3).  I don't purport 
to have a solution to (1) and (2).


Regards,
Robin




Aleksey.


On 11/8/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Pavel Pervov wrote:
> Robin,
>
> The kind of model I had in mind was along the lines of:
>> - VM maintains a linked list (or other collection type) of the
currently
>> loaded classloaders, each of which in turn maintains the collection of
>> classes loaded by that type.  The sweep of classloaders goes something
>> like:
>>
>> for (ClassLoader cl : classLoaders)
>>   for (Class c : cl.classes)
>> cl.reachable |= c.vtable.reachable
>
>
> This is not enough. There are may be live j/l/Class'es and
> j/l/Classloader's
> in the heap. Even though no objects of any classes loaded by a 
particual

> class loader are available in the heap, if we have live reference to
> j/l/ClassLoader itself, it just can't be unloaded.

OK, well how about keeping a weak reference to the j.l.ClassLoader
object instead of a strong one.  When the reference becomes (strong)ly
unreachable, invoke the class-unloading phase.

To me the key issue from a performance POV is the reachability of
classes from objects in the heap.  I don't pretend to have an answer to
the other questions---the performance critical one is the one I have
addressed, and I accept there may be many solutions to this part of the
question.

> I believe that a separate heap trace pass, different from the standard
>> GC, that visited vtables and reachable resources from there would also
>> be a viable solution.  As mentioned in an earlier post, writing 
this in


>> MMTk (where a heap trace operation is a class that you can easily
>> subtype to do this) would be easy.
>>
>> One of the advantages of my other proposal is that it can be
implemented
>> in the VM independent of the GC to some extent.  This additional
>> mark/scan phase may or may not be easy to implement, depending on the
>> structure of DRLVM GCs, which is something I haven't explored.
>
>
> DRLVM may work with (potentially) any number of GCs. Designing class
> unloading the way, which would require mark&scan cooperation from 
GC, is

> not
> generally a good idea (from my HPOV).

That's what I gathered.  hence my proposal.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University






--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Pavel Pervov wrote:

Robin,

The kind of model I had in mind was along the lines of:

- VM maintains a linked list (or other collection type) of the currently
loaded classloaders, each of which in turn maintains the collection of
classes loaded by that type.  The sweep of classloaders goes something
like:

for (ClassLoader cl : classLoaders)
  for (Class c : cl.classes)
cl.reachable |= c.vtable.reachable



This is not enough. There are may be live j/l/Class'es and 
j/l/Classloader's

in the heap. Even though no objects of any classes loaded by a particual
class loader are available in the heap, if we have live reference to
j/l/ClassLoader itself, it just can't be unloaded.


OK, well how about keeping a weak reference to the j.l.ClassLoader 
object instead of a strong one.  When the reference becomes (strong)ly 
unreachable, invoke the class-unloading phase.


To me the key issue from a performance POV is the reachability of 
classes from objects in the heap.  I don't pretend to have an answer to 
the other questions---the performance critical one is the one I have 
addressed, and I accept there may be many solutions to this part of the 
question.



I believe that a separate heap trace pass, different from the standard

GC, that visited vtables and reachable resources from there would also
be a viable solution.  As mentioned in an earlier post, writing this in
MMTk (where a heap trace operation is a class that you can easily
subtype to do this) would be easy.

One of the advantages of my other proposal is that it can be implemented
in the VM independent of the GC to some extent.  This additional
mark/scan phase may or may not be easy to implement, depending on the
structure of DRLVM GCs, which is something I haven't explored.



DRLVM may work with (potentially) any number of GCs. Designing class
unloading the way, which would require mark&scan cooperation from GC, is 
not

generally a good idea (from my HPOV).


That's what I gathered.  hence my proposal.

cheers

--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] vote on class unloading design (was Class unloading support - tested one approach)

[Robin Garner]

Pavel Pervov wrote:

Really BIG -1 from me.

As Aleksey (Ignatenko) described in original thread, j/l/Class'es and
j/l/ClassLoader's are always available in rootset, so even if no objects
of a class exist, this class will be reachable.

Actually, some sort of class unloading prototype exists in DRLVM code, 
which

implements the scheme, which is very close to what is currently voted. It
was integrated with GC v4 and is not supported by other GCs. This prototype
traces up to class loader. Robin's approach is way faster then prorotype 
is.


Unfortunately, that approach requires up to 3 GC cycles to complete in
DRLVM.


In a full-heap STW collector, my proposal would require 1 GC to collect 
unused classloaders.  In a generational STW collector, 1 full-heap GC, 
and would depend on the particular invariants enforced by an 
incremental/concurrent collector, but would be 1 complete "cycle" of any 
of the standard algorithms (I guess up to 3 GCs if the sweeps happened 
at the wrong places).



BTW, voted approach does not describe "proof-of-full-collection" algorithm
(at least I didn't find one). The only one I think of is
full-heap-collection, which _requires_ STW.


My approach simply requires the underlying collector to have a notion 
that periodically it can say that 'every reachable object allocated 
since time 't' is now marked reachable.  If the class-unloader can 
ensure that one full epoch of this invariant has passed, then it can 
safely perform unloading.



Although "automatic anloading" brings some additional requirements for GC
(weak roots (references) support and pinned allocation), it is proven to
work (patch available) and, also, is the most natural algorithm for DRLVM.


What is the run-time cost of it ?  And where is it described ?  I was 
only aware of Etienne's proposal as a full class-unloading scheme.



With the best regards,



--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Aleksey Ignatenko wrote:

Hi, Robin.
I do really like this proposed idea of marking VTables from objects via
additional word field in VTable.

But I have one question about detecting reachability of the classloaders
("sweep the vtables and check the reachability of the classloaders").
Possibly I missed something, but here is my view of the current model of
drlvm: all j.l.Classes and j.l.Classloaders are enumerated as strong roots
(strong references). Therefore we meet situation when all j.l.Classes and
j.l.Classloaders are always reachable (marked). And no sweep will help to
detect classloaders reachability.
I see the single way to distinguish if j.l.Classloader or j.l.Class was
marked not by strong root from VM but by some reference from heap - is
to write unique object value into VTable. Then we can detect if some
jlClasloader was marked from rootset (strong root from VM) or from some 
live

object.


The kind of model I had in mind was along the lines of:
- VM maintains a linked list (or other collection type) of the currently 
loaded classloaders, each of which in turn maintains the collection of 
classes loaded by that type.  The sweep of classloaders goes something like:


for (ClassLoader cl : classLoaders)
  for (Class c : cl.classes)
cl.reachable |= c.vtable.reachable

Then for any classloader where (!reachable), free its native resources 
and remove its strong root.  The java resources will be freed at next GC.



I also want to say that 1-st proposed design from me assumed addtional
mark&scan phase without enumeration of jlClasses and jlClassloaders to be
able to detect their reachability.


I believe that a separate heap trace pass, different from the standard 
GC, that visited vtables and reachable resources from there would also 
be a viable solution.  As mentioned in an earlier post, writing this in 
MMTk (where a heap trace operation is a class that you can easily 
subtype to do this) would be easy.


One of the advantages of my other proposal is that it can be implemented 
in the VM independent of the GC to some extent.  This additional 
mark/scan phase may or may not be easy to implement, depending on the 
structure of DRLVM GCs, which is something I haven't explored.


In terms of runtime cost, I would expect an auxiliary scan of this type 
to be equivalent in cost to a full-heap GC.  The other solution costs 
~1% of all GCs.  As a "back of a matchbox" calculation, if this is run 
less than every 100 (full heap) GCs, then the auxiliary trace is a win, 
if not, my other solution is a win.



Could you, please, clarify this moment.
Thanks, Aleksey.


Hope this answers your questions
cheers,
Robin



On 11/3/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:


On 11/2/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>
> >Robin, thanks for all the clarifications. Now it seems clear to me 
>and

> >I am convinced by this proposal. :-)


Yes, this proposal is the simplest and has the least perf impact. Thanks
Robin.







--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] vote on class unloading design (was Class unloading support - tested one approach)

[Robin Garner]

My proposal is purely for the performance critical operation of marking 
the classes that have live objects - as far as I'm aware, 90% of 
Etienne's design still stands.


cheers

Xiao-Feng Li wrote:

Yes, I think the discussion on class unloading is one the successful
examples of open community in coming up with a good design and making
design decision. Although the final  design can be one person's
proposal, but it's based on or enlightened by all the participants'
opinions. Eitenne's proposal is a big stride over the original one.

Cheers,
xiaofeng

On 11/8/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:

It's worth a lot, since you have implemented this in SableVM yourself.
Thanks.

On 11/7/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:
>
> For what it's worth, I'll add my humble +1.
>
> Etienne
>
> Xiao-Feng Li wrote:
> > Agreed, so plus me.
> >
> > Thanks,
> > xiaofeng
> >
> > On 11/8/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
> >
> >> On 11/7/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> >> >
> >> > All,
> >> >
> >> > It looks like the debate on class unloading has concluded.
> >> >
> >> > Let's vote on implementing Robin's proposal.
> >> >
> >> > +1
> >> >
> >> >Weldon
> >> >
> >> > PS -- When class unloading gets implemented and committed is a
> seperate
> >> > issue.
> >> >
> >> >
> >> >
> >> > +1 I think it's the best proposal among those that were discuussed
> >>
> >>
> >
>
> --
> Etienne M. Gagnon, Ph.D.http://www.info2.uqam.ca/~egagnon/
> SableVM:   http://www.sablevm.org/
> SableCC:   http://www.sablecc.org/
>
>
>





--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm][sablevm] Desing of Class Unloading Support

[Robin Garner]

Weldon Washburn wrote:

On 11/7/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:


On 07 Nov 2006 14:35:55 +0600, Egor Pasko <[EMAIL PROTECTED]> wrote:
> > I already have one idea how to benefit from movable vtables.



There would have to be a very compelling argument for making vtables
movable.  Like a business workload that Harmony needs to run within the 
next

12 months.


The cost of moving vtables would be huge.  It would have to be a very 
hefty optimization :)


--
Robin Garner
Dept. of Computer Science
Australian National University

Re: [drlvm] building jitrino in release mode

[Robin Garner]

> I've been having some problems getting some test cases to exhibit
> misbehavior for DRLVM, and it turns out that jitrino is built in release
> mode no matter what BUILD_CFG is set to.
>
> Putting this inconsistency aside for a moment, how do I get jitrino to
> build in debug?
>
> I tried changing the setting in build.sh for -Dvm.jitrino.cfg from
> 'release' to 'debug', but then DRLVM doesn't build on Ubunutu 6 with gcc
>   3.4.6.  There's some linker problem.
>
> Can anyone duplicate this (or have it build successfully in debug)?
>
> geir
>

That wouldn't be this problem, would it ?


build.native.link:
   [cc] 0 total files to be compiled.
   [cc] Starting link
   [cc] `.L217' referenced in section `.rodata' of
../_obj/jit_generic_rt_support_ia32.o: defined in discarded section
`.gnu.linkonce.t._ZN8Imm_OpndC1E9Opnd_Sizei' of
../_obj/jit_generic_rt_support_ia32.o
   [cc] `.L220' referenced in section `.rodata' of
../_obj/jit_generic_rt_support_ia32.o: defined in discarded section
`.gnu.linkonce.t._ZN8Imm_OpndC1E9Opnd_Sizei' of
../_obj/jit_generic_rt_support_ia32.o
[snip]
   [cc] `.L46' referenced in section `.rodata' of
/home/robing/tmp/harmony/harmony/working_vm/build/lnx_ia32_gcc_debug/semis/vm/jthread/_bin/libjthread.a(thread_helpers.o):
defined in discarded section
`.gnu.linkonce.t._ZN8Imm_OpndC1E9Opnd_Sizei' of
/home/robing/tmp/harmony/harmony/working_vm/build/lnx_ia32_gcc_debug/semis/vm/jthread/_bin/libjthread.a(thread_helpers.o)
   [cc] collect2: ld returned 1 exit status

--

In which case it's happening without the 'debug' setting, and I'd
appreciate some help too.  :)

robin

[drlvm] Development plan (was: Re: [drlvm] dynamic object layout)

[Robin Garner]

> We have a really difficult job to do in the next 7.5 months - to get to
> a compatible 1.0* - so I'd like to encourage people to remain as focused
> as we can to get to that point.  That doesn't mean this isn't fun, but
> the way I see it, we have a few focused months of efforts before we
> begin TCK testing, and we probably need to make some hard choices to
> delay stuff.  We're a mighty community, but a relatively small one, so
> the more of us rowing in the same direction, the better.

Is there a planned set of performance features for the 1.0 release of
harmony/DRLVM ?  I've had a quick look and can't seem to find it.  From
where I sit, reading (bits of) the mailing list, performance tweaks seem
to be thrown into the mix as people think of them rather than in a
coordinated way.  I hope I'm wrong, and that somewhere there is a plan :)

It would be good to see a list of the features that people see as being
critical to improving DRLVM's performance, and to be able to contribute to
it as a whole, rather than just on a point by point basis.

One resource I would also like to see developed is an archive of
performance feature tests, so that we can refer to an empirical record of
the cost/benefit of certain changes (for example, the cost of adding an
additional word to the object header).  Making this available, along with
patches that would allow the experiment to be reproduced seems to me like
it would be valuable.

Where this connects specifically with the 'dynamic object layout' thread
is that I was wondering what the cost of DRLVM's object model is, vs one
where the fields of all objects/arrays are at a fixed offset from the
object pointer.  This kind of object model makes adding header fields (at
the start of the object) trivial, and I'm thinking it might help
performance too.

cheers,
Robin

Re: [DRLVM][MMTk] looking for volunteers to work on this port

[Robin Garner]

Weldon Washburn wrote:

All,

It seems committing patches to drlvm and chasing down VM stability 
issues is

taking most of my time.  I would certainly welcome volunteer(s) interested
in moving the drlvm/mmtk port forward.  What needs to be done is described
in an Oct 23 email titled, "[DRLVM][MMTk] current status and plan".



I'm enthusiastic about providing assistance to whoever takes it on, but 
don't have time to take it on myself.  Besides, I think at the present 
moment it requires someone who understands DRLVM well more than an MMTk 
hacker.


Robin

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Xiao-Feng Li wrote:

Robin, good idea.

I understand the main difference between your design and Aleksey's
proposal 1 is, the tracing in your design stops as vtable, but
Aleksey's continues to classloader. On the other hand, your approach
requires an extra step to sweep the vtables in order to determine the
classloaders' reachability.


Actually there are quite a few more differences:
- This mark phase is built into the standard GC trace, like Aleksey's 
automatic class unloading proposal.
- This approach requires no additional fields in headers or objects 
(except maybe something to allow enumeration of vtables if this doesn't 
already exist)
- The additional mark comes at an extremely low cost as discussed 
previously.


The operation to sweep vtables is very cheap, and only needs to be done 
when you believe there are classloaders that can be unloaded, rather 
than at every GC.  You might for example trigger class unloading every 
time a new classloader is loaded.



If this is true, why not just let the tracing to continue as a
complete step to determine the classloaders' reachability?


Because that adds a large overhead to every GC, and requires vtables and 
classloader structures to be traced at every GC.  While the numbers of 
vtables is not large, the number of pointers to them is.  The particular 
flavour of mark in my proposal is much cheaper than the standard test 
and mark operation.



Another difference is to mark the reachability with an unconditional
write instead of a bit mask write. I think this can be applied to
either approach.


Not really.

If you use an unconditional mark, you lose the ability to test whether 
any particular mark is the first, and therefore enqueue an object for 
scanning only once, and therefore the heap trace can never complete. 
You can only use unconditional marks to process 'leaf' objects in the heap.


You can always turn a bit map into a byte map and avoid synchronized 
update, but you can't eliminate the dependent load in a standard trace 
algorithm.  The difference in performance between a load-test-write and 
a load-test-mask-write is insignificant.



Of course a separate trace of the heap is an attractive operation - in 
MMTk, this is simple to build because the transitive closure code can 
simply be subclassed (eg the sanity checker is ~250 lines of code). 
Depending on how reusable the DRLVM heap trace code is, this may or may 
not be a good option.


cheers,
Robin



Thanks,
xiaofeng

On 11/1/06, Robin Garner <[EMAIL PROTECTED]> wrote:

Actually, just thinking about how I would implement this in JikesRVM, I
would use the reachability based algorithm, but piggyback on the
existing GC mechanisms:

- Allocate a byte (or word) in each vtable for the purpose of tracking
class reachability.
- Periodically, at a time when no GC is running (even the most
aggressive concurrent GC algorithms have these, I believe), zero this
flag across all vtables.  This is the beginning of a class-unloading 
epoch.

- During each GC, when the GC is fetching the GC map for an object,
*unconditionally* write a value to the class reachability byte.  It may
make sense for this byte to be in either the first cache-line of the
vtable, or the cache line that points to the GC map - just make sure the
mark operation doesn't in general fetch an additional cache line.
- At a point in the sufficiently far future, when all reachable objects
are known to have been traced by the GC, sweep the vtables and check the
reachability of the classloaders.

The features of this approach are:

- Minimal additional work at GC time.  The additional write will cause
some additional memory traffic, but a) it's to memory that is already
guaranteed to be in L1 cache, and b) it's an unconditional independent
write, and c) multiple writes to common classes will be absorbed by the
write buffer.

- Space cost of at most 1 word per vtable.

- This works whether vtables are objects or VM structures

- If the relationship between a class and a vtable is not 1:1, this only
need affect the periodic sweep process, which should be infrequent and
small compared to a GC.

- shouldn't need a stop-the-world at any point.

I've implemented and tested the GC-relevant part of this in JikesRVM,
and the GC time overhead appears to be just under 1% in the MMTk
MarkSweep collector.

cheers,
Robin

Re: [drlvm][sablevm] Desing of Class Unloading Support

[Robin Garner]

Xiao-Feng Li wrote:

On 11/1/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:

On 01 Nov 2006 16:05:41 +0600, Egor Pasko <[EMAIL PROTECTED]> wrote:
>
> On the 0x214 day of Apache Harmony Mikhail Fursov wrote:
> > On 01 Nov 2006 15:56:28 +0600, Egor Pasko <[EMAIL PROTECTED]> 
wrote:

> > >
> > > agreed. not patching .. just reporting 'golden' VTable refs to 
GC, am

> > > I right?
> > >
> > Yes, and everytime we report it to GC and GC moves an object - it
> patches
> > the address we report.
>
> so, by saying "patching" you insist to put immediate operands into
> instructions? That's probably worth it, but it extends the JIT<->GC
> interface. How about making a simple operand (reg/mem) as the first 
step?



Egor, I thinks this is slightly more complicated problem. If vtable 
object
is moved we must update all devirtualization points in every method 
compiled

before. It can require an extension of JIT<->VM<->GC interface.
Another solution I see is to collect info about all devirtualization 
points
in JIT (code addrs) and report these addresses as enumeration roots. 
This is

JIT-only solution, and disadvantage is a significant (~hot methods count)
increase of number of objects reported.

On the other hand I see no reasons to unpin vtables in the nearest future
(Let's GC guru correct me). If you use special (freelist-type ?) 
allocator
in GC the memory fragmentation when unloading pinned vtable objects 
could be

low.


Yes, vtable should never be moved except for very weird reason. And
yes, to pin certain amount of objects is not a big performance issue
(in both temporal and spatial wise).

-xiaofeng


--
Mikhail Fursov




In MMTk, this kind of 'pinning' is an allocation-time policy decision of 
the type I was advocating in the GC helpers thread.  Once a GC allows 
for the idea of supporting multiple collection policies (which 
generational GC requires in any case), then adding a non-moving space to 
a memory manager is easy.


Most memory managers will have a non-moving large object space no matter 
 what the primary collection policy is.  The DRLVM collectors have this 
too, don't they ?


Pinning an object after allocation is a harder problem, but not 
something required in this case.


cheers

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Weldon Washburn wrote:


Its probably in the noise but it might be possible to
even reduce the overhead of clearing the vtable "mark" by using a epoch
number instead of a boolean.  The idea is that after every major GC,
increment the value used for the mark.  When sweeping the vtables, the 
stale

mark values are the unreachable classes.

cheers


Right.  I'm assuming we're all on the same page here, but I'll spell it 
out anyway:  The number of objects is orders of magnitude higher than 
the number of classes, so any operation on a 'per-class' basis can 
afford to be expensive, whereas per-object operations need to be fast.


Just looking at my stats for SpecJVM98, JBB 2000 and DaCapo 2006-10, the 
ratio of live objects to classes loaded is ~500:1 (geometric mean).  The 
extremes are 11:1 (jython) to 24000:1 (hsqldb).  These are probably also 
 very small heaps compared to enterprise workloads, which would drive 
the number of objects/class up.


The other consideration is that you are not going to want to check for 
unloadable classloaders at every GC.


Therefore, within reason, I don't think the efficiency of per-class 
operations is much of a consideration.


>  Its probably in the noise but it might be possible to
> even reduce the overhead of clearing the vtable "mark" by using a epoch
> number instead of a boolean.

Under the circumstances, requiring an additional register during GC to 
hold the class epoch number probably loses more than it gains.


cheers

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Rana Dasgupta wrote:

Robin,
   The basic difference of this with Etienne's method is that the flag is
on the vtable, instead of the CL instance. Do I understand correctly ? The
GC perf impact is therefore reduced because you need to lookup
object->vtable instead of object->class->CLinstancewhen tracing the heap.
The space overhead is correspondingly slightly higher. Also the GC impact
may look lower because there are a couple of pseudo GC cycles to reset the
vtables and sweep the vtables.

Thanks,
Rana


The relevant part of Etienne's design I believe is this:


7- Each class loader structure maintains a set of boolean flags, one
 flag per "non-nursery" garbage collected area (even when thread-local
 heaps are used).  The flag is set when an instance of a class loaded by
 this class leader is moved into the related GC-area.  The flag is unset
 when the GC-area is emptied, or (optionally) when it can be determined
 that no instance of a class loaded by this class loader remains in the
 GC-area.  This is best implemented as follows: a) use an unconditional
 write of "true" in the flag every time an object is moved into the
 GC-area by the garbage collector, b) unset the related flag in "all"
 class loader structures just before collecting a GC-area, then setting
 the flag back when an object survives in the area.


My design differs in several key ways from this:
1. There is no requirement for a per non-nursery area flag
2. The mark byte/word is set unconditionally whenever an object is 
visited by the GC, not when an object is moved into a particular mature 
space.  This may be the same for some GCs, but not all.
3. The mark byte/word is an unconditional write - Etienne's proposal 
would use a load/mask/write sequence.  This is performance critical.
4. My memory of x86 assembler is a little rusty, but I believe a 
constant store can be done without requiring a register for the value to 
be written, where as or-ing a bit value into a word requires a temporary 
register or two.
5. In a parallel GC, setting bits in a mask requires a synchronized 
update.  My design doesn't.


The point is an unconditional store to a structure you are already 
accessing is very cheap, whereas register spills, loads and synchronized 
updates are expensive.


cheers


On 10/31/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Actually, just thinking about how I would implement this in JikesRVM, I
would use the reachability based algorithm, but piggyback on the
existing GC mechanisms:

- Allocate a byte (or word) in each vtable for the purpose of tracking
class reachability.
- Periodically, at a time when no GC is running (even the most
aggressive concurrent GC algorithms have these, I believe), zero this
flag across all vtables.  This is the beginning of a class-unloading
epoch.
- During each GC, when the GC is fetching the GC map for an object,
*unconditionally* write a value to the class reachability byte.  It may
make sense for this byte to be in either the first cache-line of the
vtable, or the cache line that points to the GC map - just make sure the
mark operation doesn't in general fetch an additional cache line.
- At a point in the sufficiently far future, when all reachable objects
are known to have been traced by the GC, sweep the vtables and check the
reachability of the classloaders.

The features of this approach are:

- Minimal additional work at GC time.  The additional write will cause
some additional memory traffic, but a) it's to memory that is already
guaranteed to be in L1 cache, and b) it's an unconditional independent
write, and c) multiple writes to common classes will be absorbed by the
write buffer.

- Space cost of at most 1 word per vtable.

- This works whether vtables are objects or VM structures

- If the relationship between a class and a vtable is not 1:1, this only
need affect the periodic sweep process, which should be infrequent and
small compared to a GC.

- shouldn't need a stop-the-world at any point.

I've implemented and tested the GC-relevant part of this in JikesRVM,
and the GC time overhead appears to be just under 1% in the MMTk
MarkSweep collector.

cheers,
Robin

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

> Interesting idea!   It seems the real issue is "marking and sweeping" the
> vtables.  A stab at categorizing the approaches:
>
> a)
> Force vtables to be as similar to ordinary java objects as possible.  The
> upside: existing GC algorithms will work unaltered.  The downside is
> vtables
> of vtables of vtables...  This has already been discussed at length.
>
> b)
> Force vtables to live and die in a unique "vtable space".  The big
> challenge seems to be building a custom GC algorithm that is simpler and
> less invasive than doing a) above.  Most likely the performance of the
> custom GC algorithm will never be an issue.  Vtables have some very
> interesting properties that may make this doable.  The 4 (or 8) bytes at
> offset "K" always point to a class structure which, in turn, always has a
> pointer at offset "Z" back to the vtable.  There are way fewer vtables
> than
> objects.  For practical reasons, it can be assumed that vtables will
> always
> be pinned.  The number of class structs/objects is no greater than the
> number of vtables.

I guess ... the issue of where vtables and other classloader related
structures lives seems to me to be simple engineering.  Whether they are
malloced in the C heap or 'new'ed in an immovable Java heap makes little
difference.  After all they are very very small compared to the rest of
the heap.

> A half-baked scheme that might be good enough:  Partition off 50 megabytes
> as a hard, fixed vtables space.  Then do a word-by-word scan to pick up
> candidate pointers class structs.  Then filter the candidate class struct
> pointers by verifying the back pointers.  Occasionally there might be
> "floating garbage" with this approach but a valid, live vtable should
> never
> be accidentally freed.  The filtered set are the "live" vtables.  Next
> scan
> the live vtables looking for members that were never marked by the regular
> GC as mentioned  below.  When found, zero the vtable, link-list on a free
> vtable space list, mark the class struct as "vtable-less".  Process the
> "vtable-less" class struct, etc...

This sounds a bit complicated - i'm sure it could be done by maintaining
linked lists of all the classes loaded by each classloader (pointing to
vtables from there) and traversing it.  Traverse this once, propagating
the mark bits upwards and you are done.

> It seems as long as part of the system is built using garbage collected
> java
> objects and part of the system is built using malloc/free C structs, the
> problem of releasing connected objects/C_structs will be messy and hacky.
> In a sense, this issue is really a motivation for re-writing the whole VM
> in
> Java... hmmm...

Well in Java-in-Java you would hardly want to do a full trace operation
for every vtable pointer - that would be quite costly imo, because in a
full gc you do a test-and-set, then conditionally copy and/or enqueue for
scanning.   So there's a dependent load and conditional store, and the
test to detect the gc policy for the vtable space.

And you really don't want to have moveable vtables.  The cost of updating
the forwarding pointers alone would be a killer, let alone the engineering
difficulties.

There are definitely reasons to write a VM in java, but I don't believe
this is one of them!  As I say below I wouldn't do class unloading on top
of the standard GC mechanism - I would do it in the VM, using a hook from
the GC's tracing.

>
> On 10/31/06, Robin Garner <[EMAIL PROTECTED]> wrote:
>>
>> Actually, just thinking about how I would implement this in JikesRVM, I
>> would use the reachability based algorithm, but piggyback on the
>> existing GC mechanisms:
>>
>> - Allocate a byte (or word) in each vtable for the purpose of tracking
>> class reachability.
>> - Periodically, at a time when no GC is running (even the most
>> aggressive concurrent GC algorithms have these, I believe), zero this
>> flag across all vtables.  This is the beginning of a class-unloading
>> epoch.
>> - During each GC, when the GC is fetching the GC map for an object,
>> *unconditionally* write a value to the class reachability byte.  It may
>> make sense for this byte to be in either the first cache-line of the
>> vtable, or the cache line that points to the GC map - just make sure the
>> mark operation doesn't in general fetch an additional cache line.
>> - At a point in the sufficiently far future, when all reachable objects
>> are known to have been traced by the GC, sweep the vtables and check the
>> reachability of the classloaders.
>>
>> The features of this approach are:
>>
>> - Minimal additional wor

Re: [drlvm] Class unloading support - tested one approach

[Robin Garner]

Actually, just thinking about how I would implement this in JikesRVM, I 
would use the reachability based algorithm, but piggyback on the 
existing GC mechanisms:


- Allocate a byte (or word) in each vtable for the purpose of tracking 
class reachability.
- Periodically, at a time when no GC is running (even the most 
aggressive concurrent GC algorithms have these, I believe), zero this 
flag across all vtables.  This is the beginning of a class-unloading epoch.
- During each GC, when the GC is fetching the GC map for an object, 
*unconditionally* write a value to the class reachability byte.  It may 
make sense for this byte to be in either the first cache-line of the 
vtable, or the cache line that points to the GC map - just make sure the 
mark operation doesn't in general fetch an additional cache line.
- At a point in the sufficiently far future, when all reachable objects 
are known to have been traced by the GC, sweep the vtables and check the 
reachability of the classloaders.


The features of this approach are:

- Minimal additional work at GC time.  The additional write will cause 
some additional memory traffic, but a) it's to memory that is already 
guaranteed to be in L1 cache, and b) it's an unconditional independent 
write, and c) multiple writes to common classes will be absorbed by the 
write buffer.


- Space cost of at most 1 word per vtable.

- This works whether vtables are objects or VM structures

- If the relationship between a class and a vtable is not 1:1, this only 
need affect the periodic sweep process, which should be infrequent and 
small compared to a GC.


- shouldn't need a stop-the-world at any point.

I've implemented and tested the GC-relevant part of this in JikesRVM, 
and the GC time overhead appears to be just under 1% in the MMTk 
MarkSweep collector.


cheers,
Robin

Re: [drlvm] Class unloading support

[Robin Garner]

Weldon Washburn wrote:

On 10/30/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Weldon Washburn wrote:
> On 10/27/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>
>>
>>
>> Weldon Washburn wrote:
>> > Steve Blackburn was in Portland Oregon today.  I mentioned the idea
of
>> > adding a  reference pointer from object to its j.l.Class instance.
>> MMTk
>> > was
>> > not designed with this idea in mind.  It looks like you will need to
>> fix
>> > this part of MMTk and maintain it yourself.  Steve did not seem
>> thrilled
>> at
>> > adding this support to MMTk code base.

Actually I think the answer may have been a little garbled along the way
here: MMTk is not a memory manager *for* Java, it is simply a memory
manager.  We have been careful to eliminate language-specific features
in the heap that it manages.  MMTk has been used to manage C# (in the
Rotor VM) and was being incorporated into a Haskell runtime until I ran
out of time.

Therefore, MMTk knows nothing about the concept of class unloading, or
java.lang.Class.

>> How does MMTk support class unloading then?
>
>
> MMTk has no special support for class unloading.  This may have
> something to
> do with the entire system being written in Java thus class unloading
come
> along for free.  If there needs to be a modification to support special
> case
> objects in DRLVM, someone will need to fixup MMTk and provide onging
> support of this patch in Harmony.  I have zero idea how big this effort
> would be.   It would also be good to hear what the impact will be on
GCV5.

MMTk implements several algorithms for retaining the reachable objects
in a graph and recycling space used by unreachable ones.  It relies on
the host VM to provide a set of roots.  It supports several different
semantics of 'weak' references, including but not confined to those
required by Java.

If you can implement class unloading using those (which the current
proposal does), then MMTk can help.

If you want to put a pointer to the j.l.Class in the object header, MMTk
will not care, as it has no way of knowing.  If you put an additional
pointer into the body of every object, then MMTk will see it as just
another object to scan.

Remember MMTk is a memory manager, not a Java VM!


Conversely, supporting some exotic class unloading mechanism in MMTk
shouldn't be hard and wouldn't deter me from trying it out.




Robin, it would be great if you can get MMTk to support this class 
unloading

effort.  My main concern is the ongoing maintenance of MMTk class unloading
support.


I haven't seen any proposal that requires MMTk to be modified, so it's a 
moot point at the moment.



A question for all involved.  Is it possible to somehow make it appear that
the new objects related to unloading  (VTable, ClassLoader, etc)  are 
always
reachable and thus never collected?  I am trying to figure out a way to 
make

integration of class unloading independent of correct support inside the GC
and JIT.  This option could be a command line switch or compile time 
option.


Simple.  Keep a list or table of these objects as part of the root set. 
 Enumerate it optionally depending on a command line option.


cheers,
Robin

Re: [drlvm] Class unloading support

[Robin Garner]

Ivan Volosyuk wrote:

Robin, thank you for this information. I want to ask a few questions
to check that I understand you correctly.

On 10/31/06, Robin Garner <[EMAIL PROTECTED]> wrote:

MMTk implements several algorithms for retaining the reachable objects
in a graph and recycling space used by unreachable ones.  It relies on
the host VM to provide a set of roots.  It supports several different
semantics of 'weak' references, including but not confined to those
required by Java.

If you can implement class unloading using those (which the current
proposal does), then MMTk can help.

If you want to put a pointer to the j.l.Class in the object header, MMTk
will not care, as it has no way of knowing.  If you put an additional
pointer into the body of every object, then MMTk will see it as just
another object to scan.


Does this mean that MMTk will not work with VM in which VTable pointer
(a pointer in object header) points to other heap object?


If the GC map for the object includes this pointer, MMTk will trace it, 
otherwise not.MMTk's view of an object is abstracted through the 
implementation-specific ObjectModel interface, which provides isolation 
from the implementation details.  When I talk about 'object header', 
more precisely I'm talking about the fields that MMTk doesn't see, since 
MMTk has no real concept of an object header.


In JikesRVM, the TIB is actually an Object[] that lives in the heap - we 
don't trace TIBs from objects, but (AFAIR) via roots from the VM.  If 
you want to trace them during GC, just give MMTk GC maps that include 
them, and it will.  The invariant is simply that the ObjectModel must be 
able to understand the vtables.




Remember MMTk is a memory manager, not a Java VM!


Conversely, supporting some exotic class unloading mechanism in MMTk
shouldn't be hard and wouldn't deter me from trying it out.  If (as a
wild idea) you wanted to periodically scan the heap, and count all
references to each classloader, you could implement this with very
little work as a TraceLocal object, and then extend the GC plan you
wanted with an additional GC phase that would periodically do one of
these scans after a major GC (for example).


This looks similar to approach #2 discussed here, agree?



If what you mean is Aleksey's 'Mark and scan' proposal, yes, that sounds 
right.  I'm not advocating it as 'the solution' because I don't know 
what's best here, just saying that implementing it in MMTk wouldn't 
necessarily be hard.


cheers

Re: [drlvm] Class unloading support

[Robin Garner]

Weldon Washburn wrote:

On 10/27/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:




Weldon Washburn wrote:
> Steve Blackburn was in Portland Oregon today.  I mentioned the idea of
> adding a  reference pointer from object to its j.l.Class instance.  
MMTk

> was
> not designed with this idea in mind.  It looks like you will need to 
fix
> this part of MMTk and maintain it yourself.  Steve did not seem 
thrilled

at
> adding this support to MMTk code base.


Actually I think the answer may have been a little garbled along the way 
here: MMTk is not a memory manager *for* Java, it is simply a memory 
manager.  We have been careful to eliminate language-specific features 
in the heap that it manages.  MMTk has been used to manage C# (in the 
Rotor VM) and was being incorporated into a Haskell runtime until I ran 
out of time.


Therefore, MMTk knows nothing about the concept of class unloading, or 
java.lang.Class.



How does MMTk support class unloading then?



MMTk has no special support for class unloading.  This may have 
something to

do with the entire system being written in Java thus class unloading come
along for free.  If there needs to be a modification to support special 
case

objects in DRLVM, someone will need to fixup MMTk and provide onging
support of this patch in Harmony.  I have zero idea how big this effort
would be.   It would also be good to hear what the impact will be on GCV5.


MMTk implements several algorithms for retaining the reachable objects 
in a graph and recycling space used by unreachable ones.  It relies on 
the host VM to provide a set of roots.  It supports several different 
semantics of 'weak' references, including but not confined to those 
required by Java.


If you can implement class unloading using those (which the current 
proposal does), then MMTk can help.


If you want to put a pointer to the j.l.Class in the object header, MMTk 
will not care, as it has no way of knowing.  If you put an additional 
pointer into the body of every object, then MMTk will see it as just 
another object to scan.


Remember MMTk is a memory manager, not a Java VM!


Conversely, supporting some exotic class unloading mechanism in MMTk 
shouldn't be hard and wouldn't deter me from trying it out.  If (as a 
wild idea) you wanted to periodically scan the heap, and count all 
references to each classloader, you could implement this with very 
little work as a TraceLocal object, and then extend the GC plan you 
wanted with an additional GC phase that would periodically do one of 
these scans after a major GC (for example).


cheers

Re: [DRLVM][MMTk] current status and plan

[Robin Garner]

Weldon Washburn wrote:

On 10/25/06, Robin Garner <[EMAIL PROTECTED]> wrote:


>follows.  Comments, suggestions are welcome.   It would be much
>appreciated if Steve Blackburn and Robin Garner would reply to the
> questions
>below directed to the "MMTk guys".

Interleaved below.

>  - prepareCollector() method – Its not clear MMTk/DRLVM
>  needs to do anything. (MMTk folks please comment on what the
> VM is supposed
>  to do!)

The CollectorContext object is the per-thread context object for each GC
thread.  A uniprocessor, single-threaded java program will have one
MutatorContext and one CollectorContext.  A uniprocessor multithreaded
java program (ie all practical programs on a uniprocessor) will have one
CollectorContext and several MutatorContexts.

This method is provided so that the VM can do whatever it requires to a
collector thread.  If this is nothing (which in drlvm it could well be),
this can be a no-op.  Whether that is true or not requires DRLVM 
internals

knowledge I don't have.



Can we simply hardwire the number of CollectorContexts to be one?  At a
later date when this becomes a performance problem on a 4-way SMP, we can
fix this problem.  What do you think?


For the purposes of initial implementation, sure.  But I'd be inclined 
to debug a parallel collector earlier than that.




>  - scheduleFinalizerThread() – do nothing at this stage (It
>  will need to be fixed when MMTk/DRLVM is capable of running
> workloads that
>  need finalizers.)
>   - Lock class
>  - This looks complete.  (Can MMTk folks take a look and
>  confirm?)

Lets run through this together on Thursday.



Well, we did not get to this today in Portland Oregon.  Maybe you can tell
me what needs to be done over harmony-dev.


I'll have a look in SVN and see what I can do.

>  - getObjectType() returns an object of type MMTtype.  
Currently

>  there is a very simple cache of MMType objects.  We need
>  to confirm this approach is functionally correct (MMTk guys,
> please
>  comment).  Then determine if a simple cache is good enough
>  to bring up work loads such as Dacapo and SpecJBB.  A
>  design issue that needs to be resolved – what part of the
> MMTk heap should
>  MMType objects be allocated from?  Maybe ImmortalSpace
>  (MMTk guys, is this correct?)

When a class is loaded, you should request an MMType from MMTk, then 
store

it somewhere that it can be returned by this call.  Allocating in
ImmortalSpace would be appropriate (or move before each GC using the
preCopyGCInstances hook).  JikesRVM does this (immortal allocation) using
compile-time allocation policy based on call site (by type) and object
type.



I think this means that MMTk has a requirement on JIT functionality.  That
is, the JIT must support compile-time allocation policy based on callsite.
Is this correct?


I don't think it *requires* compile-time allocation policy.  As I say 
above, if you use preCopyGCInstances to move the MMType objects before 
GC, you should be able to allocate them in the general Java heap.  This 
goes for all the MMTk objects (xxContexts, Phases etc).


But VM/Compiler control over allocation policy is certainly very Nice To 
Have for a variety of reasons.



   - Project 5
>   - Debug and verify JIT support for MMTk's  "Uninterruptible"
>   class.  This basically means that the JIT needs to not insert GC
>   polling calls when JITing an MMTk class that extends
> "Uninterruptible".
>   This project depends on VM and JIT support for Back-branch
>   polling.  It probably does not need to be fully developed and
>   debugged until we try to run multithread java apps.  The reason
>   is because it requires two or more running Java thread to create
> a condition
>   where one thread want to arbitrarily suspend the other java
threads
> at GC
>   safepoints.

My feeling is multithreading is so fundamental to Java that this needs to
be done pdq.  I'm not sure how the helpers for GCV[45] are going to cope
without it.



Sorry for not being clear.  Yes, multithreading is fundamental to drlvm and
it works.  I simply want to turn it off for this stage of the MMTk/DRLVM
port.  Back-branch polling and Uninterruptible have to work for the ongoing
DRLVM vmmagic vm helper coding that is currently under way.

Re: [drlvm] Calling native methods from Java code: implementation details

[Robin Garner]

I thought moxie already had an implementation of fast native calls ? 
Perhaps you could look at that ?


cheers,
Robin

Mikhail Fursov wrote:

On 10/23/06, Pavel Afremov <[EMAIL PROTECTED]> wrote:


Mikhail could you summarize all issues which should be clarified and
possible solutions for these issues?



Ok, I'm going to summarize our discussion. Correct me if I missed 
something.


Problem 1A:  How JIT will know if a Java method must be replaced with a
direct native call.
Solution: Check if the class implements special marker interface or have an
annotation. The class must be loaded by bootstrap classloader.

Problem 1B: How JIT will know which VM helper must be replaced with a
fast-path Java version.
Solution: Check if predefined VM property is set. Get the name of the class
from the property. Initiate the resolution and initialization of the class
with fast-path helper.

Problem 2: How JIT will get the address of the native method to generate a
direct call?
Here we have 2 solutions:
Solution 2.1: Every component must have C method to return address for a
given Java method 'void* get_direct_address(Method_Handle)'. The name of 
the

component is stored as annotation for the method.
Solution 2.2:  Every class with a method to be replaced with direct native
call must have a JNI method: 'static native long get_address(int id). JIT
asks VM to get the address for a given method handle. VM gets the id from
the method annotations and calls the JNI method to derive the address.



Problem 3: How JIT will know which calling convention to use?
Solution 3: Use method's annotation again.



The current status:
Today we need only solution for 1B problem. I've implemented the first
fast-path vmhelper and inlined it using the solution described above. The
patch depends on JIRA1942 and JIRA1949. I'm going to commit the patch after
these JIRAs are accepted.

We have multiple solution for the problem 2. We can postpone the decision
for a awhile. The problem is not critical for the vmhelper inlining task .

Re: [drlvm][build] Where to put mmtk.jar?

[Robin Garner]

I'll need to consult within the MMTk world, but my preference would be to
distribute jar files for the vmmagic and mmtk interfaces, separate to the
implementation.

Leave it with me.

cheers

Geir Magnusson Jr. wrote:
> works for me
>
> Weldon Washburn wrote:
>>
>>
>> On 10/26/06, *Geir Magnusson Jr.* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
>>
>>
>>
>> Mikhail Fursov wrote:
>>  > Folks,
>>  > I'm preparing the first patch with VM helpers inlining framework
>>  > implementation.
>>  > The Java fast-path helpers use org.vmmagic.unboxed classes (or
>> "magic"
>>  > classes ) to use address arithmetic in helpers. That is the
>> mmtk.jar is
>>  > required to build these classes.
>>  > The problem is that I do not know what is the best place to put
>> mmtk.jar.
>>  > Should we make it downloadable from some 3rd party page? My vote
>> is -1
>>  > here.
>>
>> Why not?  We do that for everything else, where we can.
>>
>> I agree with Geir.  Its best to try to stay out of the business of
warehousing 3rd party stuff.  Let's download MMTk.jar from a web page
during "build fetch-depends".   We can change this if it causes
problems. Robin Garner, can you help us with this?  What is the best
URL for this?
>>
>>  > Let's be careful about what is in the JAR.
>>  > Could we put it directly into SVN? I'm +1 here. We have to decide
>> only
>>  > about
>>  > a folder name.
>>  >
>>  > Any thoughts?
>>  >
>>
>>
>>
>>
>> --
>> Weldon Washburn
>> Intel Enterprise Solutions Software Division

Re: [DRLVM][MMTk] current status and plan

[Robin Garner]

>follows.  Comments, suggestions are welcome.   It would be much
>appreciated if Steve Blackburn and Robin Garner would reply to the
> questions
>below directed to the "MMTk guys".

Interleaved below.

>   - The next step is to integrate MMTk in the early DRLVM boot
>   process.  The goal is to make sure all code the JIT generates
>   will allocate out of the MMTk heap.  This is a "chicken and egg"
>   kind of problem since no JITed code can execute until DRLVM has
> a GC that is
>   ready to support object allocation.  Most likely we will use the
>   MMTk notion of "ImmortalSpace" for early object allocation.  Objects
>   in ImmortalSpace are never collected, never moved.  At this
>   stage of MMTk/DRLVM porting, the cost of dead uncollected objects
> wasting
>   ImmortalSpace memory is not a concern.

The ability to specify the allocator on a per-call-site and per-type basis
can be useful here.

>   - Collection Class
>  - triggerCollection() method needs to be connected to the
>  Java API that forces a GC (this is low priority)
>  - prepareMutator() method probably needs to be integrated
>  with back-branch polling mechanism.  Also need to confirm
>  the requirement that a thread suspend request does indeed
> force the target
>  thread to be suspended at a GC safepoint.  (MMTk guys, can
>  you confirm this?)

This method is called at the start of a collection for every mutator
context that the VM interface knows about.  The work is farmed out in
parallel between the available collector threads.  What exactly happens is
up to each client VM, but this is the chance that the VM gets to prepare
each mutator thread for having its roots enumerated.

MMTk imposes no particular constraints per se on where the VM suspends its
threads, and essentially provides this method to allow the VM to do
whatever is required to suspend mutator threads and/or advance them to a
safe point before their roots are enumerated.


>  - prepareCollector() method – Its not clear MMTk/DRLVM
>  needs to do anything. (MMTk folks please comment on what the
> VM is supposed
>  to do!)

The CollectorContext object is the per-thread context object for each GC
thread.  A uniprocessor, single-threaded java program will have one
MutatorContext and one CollectorContext.  A uniprocessor multithreaded
java program (ie all practical programs on a uniprocessor) will have one
CollectorContext and several MutatorContexts.

This method is provided so that the VM can do whatever it requires to a
collector thread.  If this is nothing (which in drlvm it could well be),
this can be a no-op.  Whether that is true or not requires DRLVM internals
knowledge I don't have.

>  - rendezvous() method current is "hacked" to support only
>  a single thread Java app.  This needs to be fixed.  Its
>  not critical until we need to support multithread GC apps.

This is a pretty standard barrier synchronization construct - shouldn't be
hard to implement.

>  - scheduleFinalizerThread() – do nothing at this stage (It
>  will need to be fixed when MMTk/DRLVM is capable of running
> workloads that
>  need finalizers.)
>   - Lock class
>  - This looks complete.  (Can MMTk folks take a look and
>  confirm?)

Lets run through this together on Thursday.

>   - Memory Class
>  - This looks compete except for the large 450MB byte array
>  that is allocated from existing DRLVM GC.  This "hack"
>  will need to be removed (see below) to integrate MMTk into
> early stage DRLVM
>  boot process.  (Can MMTK folks confirm this analysis?)

MMTk should be allocating memory directly through mmap.

>   - ObjectModel Class
>  - Interestingly, many methods are not called by any of the
>  initial GC algorithms  targeted (MarkSweep, SemiSpace,
>  GenMS and CopyMS).  These methods currently will execute a
>  "VM.assertions._assert(false);".  The plan is to implement
>  these methods when the assert()s are hit.  Most likely
>  this will happen when additional GC algorithms are tried.
>  - copy() implementation needs to be completed.

Should be straightforward.  Need to
a) ask the VM how large the object will be when copied (allows for hash
words etc)
b) call ((CollectorContext)this).allocCopy to allocate space, then
c) ask the VM to copy the object
d) install the forwarding pointer in the old object
e) Call ((CollectorContext)this).postCopy

>  - getObjectType() returns an object of type MMTtype.  Currently
>  there is a very simple cache of MMType o

Re: [drlvm] The first GC helper with fast-path implemented in Java: gc_alloc

[Robin Garner]

The construction of the object is probably best done by the GC itself.  VM
requests a GC context object from the GC when a new thread is created, and
calls methods on it to invoke fast path functions.

The JIT and VM can probably regard this object as totally opaque.

It's true that fast path code is somewhat constrained, it should be *what*
you do that is constrained, rather than the language constructs used to
express it.  In the MMTk fastpath code you find almost every method uses
inline/noinline pragmas, we don't use invoke interface and pay attention
to the other concerns you mention, but beyond that we pretty much write
standard Java code.  Of course it *is* carefully written to ensure that
method invocations can be devirtualised, parameters are statically
resolvable etc.  But in general we write it with clarity of expression as
the primary concern, and optimize code if it needs it.

cheers

> Robin,
>I found your last set of comments very illuminating. Thanks.
>On your suggestion below...who would manage this object, ie., construct
> and populate it, etc. before invoking the helper instance method? The jit?
> I
> think that some of the VM specific information has started leaking into
> the
> jit already! What if we want to attach a new JIT into Harmony/DRLVM
> ...what
> would be the requirements for such a jit?
>Regarding the terse Java in the fastpath code, what one can do in the
> fastpaths is somewhat constrained, isn't it? They are mostly
> uninterruptible, shouldn't raise exceptions, cause gc etc. Our java layer
> in
> the infrastructure is somewhat localized.
>
> Thanks,
> Rana
>
>
>> On 10/15/06, Robin Garner <[EMAIL PROTECTED]> wrote:
>> >
>> > Weldon Washburn wrote:
>> > > Robin,
>> > > I really like your thinking!  I would like to see Harmony drlvm
>> > support
>> > > MMTk
>> > > fully.  this may take a while.  Please feel free to keep pushing to
>> do
>> > the
>> > > right thing with the JIT.
>> >
>> > Well, with this encouragement, there are a few more things I'd like to
>> > suggest:
>> >
>> > - Rather than make TLS access be a magic, how about defining an object
>> > with fields that point to all the VM resources (such as TLS) that a
>> > helper wants, and then calling the helper as an instance method of
>> that
>> > object ?
>> >
>> >   . If devirtualisation of this is problematic for the JIT at the
>> > moment, perhaps introduce a magic pragma instead of the TLS access
>> > handler
>> >
>> > - Mikhail's prototype Java helper looks like C transliterated into
>> Java.
>> > This is reminiscent of the very early days of JikesRVM.  IMO, you
>> > should actually use Java here rather than fight it ... define an
>> > abstract Allocator class, and a concrete BumpPointer instance for
>> > example.
>> >
>> > One of the lessons of MMTk was how much you can trust the compiler,
>> and
>> > each revision uses more and more object orientation.  You guys are in
>> an
>> > ideal position, because you have control over the compiler as well as
>> > the java code.
>> >
>> > cheers
>> >
>> >
>



-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm] The first GC helper with fast-path implemented in Java: gc_alloc

[Robin Garner]

Weldon Washburn wrote:

Robin,
I really like your thinking!  I would like to see Harmony drlvm support 
MMTk

fully.  this may take a while.  Please feel free to keep pushing to do the
right thing with the JIT.


Well, with this encouragement, there are a few more things I'd like to 
suggest:


- Rather than make TLS access be a magic, how about defining an object 
with fields that point to all the VM resources (such as TLS) that a 
helper wants, and then calling the helper as an instance method of that 
object ?


  . If devirtualisation of this is problematic for the JIT at the 
moment, perhaps introduce a magic pragma instead of the TLS access handler


- Mikhail's prototype Java helper looks like C transliterated into Java. 
 This is reminiscent of the very early days of JikesRVM.  IMO, you 
should actually use Java here rather than fight it ... define an 
abstract Allocator class, and a concrete BumpPointer instance for example.


One of the lessons of MMTk was how much you can trust the compiler, and 
each revision uses more and more object orientation.  You guys are in an 
ideal position, because you have control over the compiler as well as 
the java code.


cheers


On 10/13/06, Robin Garner <[EMAIL PROTECTED]> wrote:


Mikhail Fursov wrote:
> On 10/13/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
>>
>> Why is this a problem, am I misunderstanding? Even with conventional
>> helpers, the jit needs to know the helper method signatures. We don't
>> need
>> to worry about universality of GC's. Any compatible GC will need to
>> implement  exactly the same helper fastpath contract interface as Xiao
>> Feng
>> says elsewhere.
>
>
> Yes. The only amendment from me is that JIT does not want to know if GC
is
> compatible or not. For example some GC developers may try to implement
bump
> pointer allocation by decrementing the offset in a block (like a stack)
or
> to do some cleaning like Ivan did.
> But I'm agree with you that our GCv4.1 and GC_GEN could use the same
> fast-path helper.
>
> I asked this question because of MMTk collectors. It has slightly
different
> format of the allocation sequence. If you look into the WB helpers you
see
> that the difference is even more than allocation one. IIRC the MMTk WB
> helper needs 4 params to be reported from JIT while our GC_GEN only 2.
> So we
> have to support in JIT different versions of the same helper.

As far as the MMTk 'alloc' method goes, MMTk is simply providing
flexibility that you don't need to use if you don't want to.  The Java
helper can supply default values for alignment and allocator, and MMTk
won't mind.

The barriers are slightly different.  We require the Source object,
target object and slot in order to a) be a substituting write barrier,
b) implement object-remembering barriers, c) implement reference
counting.  The fourth parameter differentiates between ASTORE, AASTORE
and PUTSTATIC barriers, and again if you only have one of these (or take
the same action in all cases), a hard-wired default will do.

If there's a really good reason you can only supply slot and target,
some of the MMTk generational collectors (eg GenMS) will work, but some
of the more interesting ones like GenRC won't.

>
>> I don't think that the jit  needs to know that an object being
>> finalizable
>> implies not to invoke the fastpath. This to me suggests that the jit
>> determines allocation policy. I don't see a problem passing the object
>> typeinfo or allocation handle to the helper. It is needed for object
init
>> anyway.
>
>
> Doing isFinalizable check during a compile time instead of runtime 
looks
> like a reasonable optimization to me.Moreover, once you want to do 
it in

> runtime, you have to add special magic isFinalizable(type). So we can
add
> such an option (configurable from the command-line or property file) to
the
> JIT.

There are reasons why the JIT might want to - if not control, then at
least influence - allocation policy.  Pretenuring is one optimization
that comes to mind.

One advantage of controlling allocation through method parameters is
that if they are statically resolvable, they are easy for the compiler
to optimize out after constant propagation.  Having the compiler
specialize code that involves (eg) looking up an allocation handle is a
much bigger ask than checking a bit in a parameter.  MMTk takes
advantage of this quite a lot - the allocation fastpath of a typical
plan may look impossibly heavy at first glance, but in most cases it
optimizes out to a handful of instructions.

Designing the interface in a way that appears like you are dynamically
checking for finalizability(is that a word?), coupled with some constant
folding and dead code elimination can give you the best of both worlds.
It al

Re: [Fwd: Re: [DRLVM][JET] write barrier for Java (mmtk)]

[Robin Garner]

Urgh.  In that case the array base offset should be specified in the
constructor of an MMType - JikesRVM doesn't have this problem, because the
object pointer always points to element 0 of an array irrespective of
header sizes and alignment issues (and hence the align and offset
parameters to 'alloc').  I was brought up to believe that a fixed offset
to the first data field was a performance win - has it been measured ?

I'll have a look at doing this in MMTk, but probably not until after OOPSLA.

cheers

> Weldon Washburn wrote:
>>> I actually stumbled on this bug during bringup.  I replaced
>>> ...plus(arrayOffset)... with ...plus(12)...  Its a hack.  But now it
>>> looks
>>> like I simply set ARRAY_BASE_OFFSET to 12 during .  Does this
>>> seem
>>> right??
>
> By the way, the array base offset may differ depending on the platform and
> array type.
> As as I know, currently on 32 bit platform (ia32) the array base offset
> is 12 for types of size <= 4.
> Double[] and long[] arrays have array base offset 16.
>
> on 64 bit platforms, if the vtable pointers are compressed (they are now),
> the array base offset is the same as on 32 bit platforms.
>
> Should someone disable vtable reference compression, the array base offset
> will be always 16 (8 bytes vptr + 4 bytes flags + 4 bytes array size).
>
>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>



-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm] The first GC helper with fast-path implemented in Java: gc_alloc

[Robin Garner]

Mikhail Fursov wrote:

On 10/13/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:


Why is this a problem, am I misunderstanding? Even with conventional
helpers, the jit needs to know the helper method signatures. We don't 
need

to worry about universality of GC's. Any compatible GC will need to
implement  exactly the same helper fastpath contract interface as Xiao
Feng
says elsewhere.



Yes. The only amendment from me is that JIT does not want to know if GC is
compatible or not. For example some GC developers may try to implement bump
pointer allocation by decrementing the offset in a block (like a stack) or
to do some cleaning like Ivan did.
But I'm agree with you that our GCv4.1 and GC_GEN could use the same
fast-path helper.

I asked this question because of MMTk collectors. It has slightly different
format of the allocation sequence. If you look into the WB helpers you see
that the difference is even more than allocation one. IIRC the MMTk WB
helper needs 4 params to be reported from JIT while our GC_GEN only 2. 
So we

have to support in JIT different versions of the same helper.


As far as the MMTk 'alloc' method goes, MMTk is simply providing 
flexibility that you don't need to use if you don't want to.  The Java 
helper can supply default values for alignment and allocator, and MMTk 
won't mind.


The barriers are slightly different.  We require the Source object, 
target object and slot in order to a) be a substituting write barrier, 
b) implement object-remembering barriers, c) implement reference 
counting.  The fourth parameter differentiates between ASTORE, AASTORE 
and PUTSTATIC barriers, and again if you only have one of these (or take 
the same action in all cases), a hard-wired default will do.


If there's a really good reason you can only supply slot and target, 
some of the MMTk generational collectors (eg GenMS) will work, but some 
of the more interesting ones like GenRC won't.




I don't think that the jit  needs to know that an object being 
finalizable

implies not to invoke the fastpath. This to me suggests that the jit
determines allocation policy. I don't see a problem passing the object
typeinfo or allocation handle to the helper. It is needed for object init
anyway.



Doing isFinalizable check during a compile time instead of runtime looks
like a reasonable optimization to me.Moreover, once you want to do it in
runtime, you have to add special magic isFinalizable(type). So we can add
such an option (configurable from the command-line or property file) to the
JIT.


There are reasons why the JIT might want to - if not control, then at 
least influence - allocation policy.  Pretenuring is one optimization 
that comes to mind.


One advantage of controlling allocation through method parameters is 
that if they are statically resolvable, they are easy for the compiler 
to optimize out after constant propagation.  Having the compiler 
specialize code that involves (eg) looking up an allocation handle is a 
much bigger ask than checking a bit in a parameter.  MMTk takes 
advantage of this quite a lot - the allocation fastpath of a typical 
plan may look impossibly heavy at first glance, but in most cases it 
optimizes out to a handful of instructions.


Designing the interface in a way that appears like you are dynamically 
checking for finalizability(is that a word?), coupled with some constant 
folding and dead code elimination can give you the best of both worlds. 
 It allows you to implement compile time optimization (in which case 
inlining, constant folding, dead code elimination etc do the work), but 
also gives you the opportunity to implement initially a slow way, and if 
there are infrequent difficult cases, leave them as runtime checks.


cheers

-- robin



-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [Fwd: Re: [DRLVM][JET] write barrier for Java (mmtk)]

[Robin Garner]

Weldon Washburn wrote:

Robin,

I did a diff of the mmtk/vm directory.  It looks like nothing really 
changed

in the vm interface code which is good.  Am I understanding this right??


That sounds about right.  And (famous last words) I think the interface 
will probably remain stable for a while now.


I did notice a change in utility/scan/mmtype.java.  
getSlot(ObjectReference,

int) returns addr.plus(VM.ARRAY_BASE_OFFSET) instead of
...plus(arrayOffset)...

I actually stumbled on this bug during bringup.  I replaced
...plus(arrayOffset)... with ...plus(12)...  Its a hack.  But now it looks
like I simply set ARRAY_BASE_OFFSET to 12 during .  Does this seem
right??


That's the intent, yes.  Each VM provides its array base offset as a 
constant.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[Fwd: Re: [DRLVM][JET] write barrier for Java (mmtk)]

[Robin Garner]

 Original Message 
Subject: Re: [DRLVM][JET] write barrier for Java (mmtk)
Date: Thu, 12 Oct 2006 11:36:33 +1000
From: Robin Garner <[EMAIL PROTECTED]>
To: harmony-dev@incubator.apache.org
References: 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]>


Weldon Washburn wrote:

Robin,

Thanks for helping clarify the issues.  The MMTk code base we are 
using is
what Steve Blackburn supplied us in mid July.  I don't know when it 
will be
suggested we move to a more recent version of MMTk.  I suspect a major 
part
of the confusion has been because of working with a code base where 
the Plan

interface is in transition.


My impression was that Steve's advance release to you included the
refactoring into MutatorContext and CollectorContext. It's been pretty 
stable (wrt major refactorings) since then.


My recollection is that writeBarrier hasn't been a method of Plan since
mid 2005 when Daniel Frampton and I refactored and created PlanLocal.
Could you double check ??

In any case, please confirm that each java thread needs to put an 
instance

of Plan in its thread-local storage and that writeBarrier() and alloc()
virtual method entry points need to be materialized from thread-local 
Plan

object.


No.  Each java thread needs a MutatorContext (or more precisely the
correct sub-type for the given plan) for each thread, and a
CollectorContext for each GC thread.  Logically, the objects Plan,
MutatorContext, CollectorContaxt, Trace and TraceLocal comprise a plan.


Also, please confirm (or deny) that we should never call
VM.barriers.performWriteInBarrier().  It only should be called by 
internal

MMTk methods (I think).


Confirmed.  Well you could call it, but it just does a store without
invoking a write barrier.  Rarely what you want outside of MMTk internals.


On 10/11/06, Robin Garner <[EMAIL PROTECTED]> wrote:


I think you must be looking at a fairly old version of MMTk.
writeBarrier is an instance method of a MutatorContext (in 
org.mmtk.plan).


MutatorContext exists to hold unsynchronized thread-local data
structures.  Particularly relevant to the write barrier, each mutator
context has its own thread-local remset.  All of the mutator context
methods of MMTk need fast access to the MMTk thread local data
structures, which is why they are instance methods.  The other critical
instance method of a MutatorContext is 'alloc', which also has it's
thread-local chunk of the space(s) it allocates into.

As far as the VM is concerned, it will be calling instance methods of a
final class.  The various classes in org.mmtk.plan.* aren't final, but
the VM interface code is expected to wrap the currently selected plan in
some final class.  JikesRVM wraps the currently selected plan classes
in a 'SelectedPlan', 'SelectedMutatorContext' etc.

As far as the VM.barriers.performWriteInBarrier() call is concerned,
the optimization required to devirtualize a call to a final method of a
static final field shouldn't be too hard to implement.  MMTk recently
moved away from using static methods for this part of the interface, to
the current abstract factory, and improved the structure of the software
significantly.  We don't want to go back!

>  I erroneously thought we could call
> VM.barriers.performWriteInBarrier() directly.  This sort of, kind of
breaks
> MMTk architecture.

well, it less 'breaks the architecture' than performs a no-op :)

-- robin

Weldon Washburn wrote:
> Ooops.  I really tangled things up.  You are right about how we are
> supposed
> to find the Java write barrier method.  It is located in
> Plan.writeBarrier().
> Each GC algorithm has a Plan class that overrides the writeBarrier()
> method.  I erroneously thought we could call
> VM.barriers.performWriteInBarrier() directly.  This sort of, kind of
breaks
> MMTk architecture.  By design, each GC algorithm in MMTk is 
supposed to

> call
> Plan.writeBarrier() which, in turn, will call
> VM.barriers.performWriteInBarrier.
>
> Sorry for the confusion.
>
>
>
>
> On 10/10/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
>>
>> Yes, we can run the usual inliner after helpers are inlined.
>> The only problem I want to notice is that once we have different
helpers
>> for
>> different GCs it's a bad idea to use virtual method calls in
performance
>> sensitive helpers. You are allowed to do it, but the better solution
>> is to
>> teach the helper to use a final implementation of the Barrier and
replace
>> the helper once the implementation of the Barrier class is changed.
>>
>> On 10/11/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
>> >

[Fwd: Re: [DRLVM][JET] write barrier for Java (mmtk)]

[Robin Garner]

 Original Message 
Subject: Re: [DRLVM][JET] write barrier for Java (mmtk)
Date: Thu, 12 Oct 2006 11:18:20 +1000
From: Robin Garner <[EMAIL PROTECTED]>
To: harmony-dev@incubator.apache.org
References: 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]> 
<[EMAIL PROTECTED]>


My standard practice, at least at this early stage of development, is to
try to track SVN head as far as possible.  When you get close to
release, sure, freeze versions.  It's a process choice, and up to you.

I realise there might be issues with you accessing svn head directly - 
I'm happy to cut you a new jar file whenever you like.  The current 
version is at http://cs.anu.edu.au/people/Robin.Garner/mmtk-20061012.jar


AFAIK, Steve's private release to you was an *advance* release, that SVN
trunk has now overtaken (he didn't want to commit it until after the
JikesRVM 2.4.5 release).

More in another reply.

Weldon Washburn wrote:
hmm we may have version skew going on here.  Mikhail, Robin can we 
stay

with the July 14 (or there abouts) version that Steve Blackburn posted to
his web page?

On 10/11/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:


On 10/11/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
>
> Robin,
>
> Thanks for helping clarify the issues.  The MMTk code base we are 
using

is
> what Steve Blackburn supplied us in mid July.


Looks like 'unboxed' package from MMTk has been updated recently too. 
I've

found 'Address::prefetchNTA()' op today we do not have in our version.


On ia62/x64 systems, prefetch and prefetchNTA should be synonymous.





--
Mikhail Fursov









-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM][JET] write barrier for Java (mmtk)

[Robin Garner]

I think you must be looking at a fairly old version of MMTk. 
writeBarrier is an instance method of a MutatorContext (in org.mmtk.plan).


MutatorContext exists to hold unsynchronized thread-local data 
structures.  Particularly relevant to the write barrier, each mutator 
context has its own thread-local remset.  All of the mutator context 
methods of MMTk need fast access to the MMTk thread local data 
structures, which is why they are instance methods.  The other critical 
instance method of a MutatorContext is 'alloc', which also has it's 
thread-local chunk of the space(s) it allocates into.


As far as the VM is concerned, it will be calling instance methods of a 
final class.  The various classes in org.mmtk.plan.* aren't final, but 
the VM interface code is expected to wrap the currently selected plan in 
 some final class.  JikesRVM wraps the currently selected plan classes 
in a 'SelectedPlan', 'SelectedMutatorContext' etc.


As far as the VM.barriers.performWriteInBarrier() call is concerned,
the optimization required to devirtualize a call to a final method of a 
static final field shouldn't be too hard to implement.  MMTk recently 
moved away from using static methods for this part of the interface, to 
the current abstract factory, and improved the structure of the software 
significantly.  We don't want to go back!


>  I erroneously thought we could call
> VM.barriers.performWriteInBarrier() directly.  This sort of, kind of 
breaks

> MMTk architecture.

well, it less 'breaks the architecture' than performs a no-op :)

-- robin

Weldon Washburn wrote:
Ooops.  I really tangled things up.  You are right about how we are 
supposed
to find the Java write barrier method.  It is located in 
Plan.writeBarrier().

Each GC algorithm has a Plan class that overrides the writeBarrier()
method.  I erroneously thought we could call
VM.barriers.performWriteInBarrier() directly.  This sort of, kind of breaks
MMTk architecture.  By design, each GC algorithm in MMTk is supposed to 
call

Plan.writeBarrier() which, in turn, will call
VM.barriers.performWriteInBarrier.

Sorry for the confusion.




On 10/10/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:


Yes, we can run the usual inliner after helpers are inlined.
The only problem I want to notice is that once we have different helpers
for
different GCs it's a bad idea to use virtual method calls in performance
sensitive helpers. You are allowed to do it, but the better solution 
is to

teach the helper to use a final implementation of the Barrier and replace
the helper once the implementation of the Barrier class is changed.

On 10/11/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
>
> Makes sense, using a standard barrier invocation fastpath. But I assume
> that
> the MMTk WB helper that it will call needs to be inlined too.
>
> Thanks
>
>
> On 10/10/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
> >
> > Weldon,
> > > I thought about slightly different approach.
> > > Why not to write fast-path VM helper like was proposed in the 
thread

> > > "[drlvm]Extending..."
> > > This helper (a static method) can be inlined by JIT without any
> > > devirtualization and call any method needed from MMTk or native
> > > implementation. So JIT won't know if it works with MMTk or with a
> native
> > > GC:
> > > all you need is just to replace the Java version of the helper.
> > > ?
> > >
> >
> >
>
>


--
Mikhail Fursov








-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [doc] new "Getting Started" guides

[Robin Garner]

Robin Garner wrote:

Vladimir Gorr wrote:

On 9/22/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:


Since Harmony is an open source project probably we should change the
link from commercial WinZip to open source 7-Zip[1] :)



It has *GNU LGPL. *IIRC we can't use this, can it?

Thanks,
Vladimir.


Isn't info-zip the standard free distribution ?  It has an essentially 
public domain license afaict.


Correction: BSD-style.




SY, Alexey


1. http://www.7-zip.org/

2006/9/22, Geir Magnusson Jr. <[EMAIL PROTECTED]>:
> As discussed earlier today, there are now two new "Getting Started"
> guides on the website, accessible from the homepage.
>
>http://incubator.apache.org/harmony/quickhelp_users.html
>
>http://incubator.apache.org/harmony/quickhelp_contributors.html
>
> There is still more work to do - for example, we need to fill in the
> lists of tools needed and dependencies.  (I'll add a fresh Ubuntu VM
> in Parallels tomorrow and work though all the deps that need to be
> added)
>
> Give a read, test it out, and comment...
>
> geir
>
>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>


--
Alexey A. Petrenko
Intel Middleware Products Division

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]







-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [doc] new "Getting Started" guides

[Robin Garner]

Vladimir Gorr wrote:

On 9/22/06, Alexey Petrenko <[EMAIL PROTECTED]> wrote:


Since Harmony is an open source project probably we should change the
link from commercial WinZip to open source 7-Zip[1] :)



It has *GNU LGPL. *IIRC we can't use this, can it?

Thanks,
Vladimir.


Isn't info-zip the standard free distribution ?  It has an essentially 
public domain license afaict.



SY, Alexey


1. http://www.7-zip.org/

2006/9/22, Geir Magnusson Jr. <[EMAIL PROTECTED]>:
> As discussed earlier today, there are now two new "Getting Started"
> guides on the website, accessible from the homepage.
>
>http://incubator.apache.org/harmony/quickhelp_users.html
>
>http://incubator.apache.org/harmony/quickhelp_contributors.html
>
> There is still more work to do - for example, we need to fill in the
> lists of tools needed and dependencies.  (I'll add a fresh Ubuntu VM
> in Parallels tomorrow and work though all the deps that need to be
> added)
>
> Give a read, test it out, and comment...
>
> geir
>
>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>


--
Alexey A. Petrenko
Intel Middleware Products Division

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]







-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] Build problems - Stl.h

[Robin Garner]

Egor Pasko wrote:

On the 0x1EC day of Apache Harmony Robin Garner wrote:

Thanks to everyone who assisted before ... the build is progressing,
but now fails with


   [cc] /usr/lib/gcc/i486-linux-gnu/4.0.3/../../../../include/c++/4.0.3/bits/stl_tree.h:36 7: 
error: no matching function for call to ‘Jitrino::StlMMAllocator::construct 
(std::pair*, const std::pair&)’
   [cc] /home/robing/harmony/drlvm/vm/jitrino/src/shared/Stl.h:123: note: candidates 
are: void Jitrino::StlMMAllocator::construct(T*, const T&) [with T = 
Jitrino::Opnd*]
BUILD FAILED
/home/robing/harmony/drlvm/build/make/build.xml:404: The following error 
occurred while execut ing this line:
/home/robing/harmony/drlvm/build/make/build.xml:411: The following error 
occurred while execut ing this line:
/home/robing/harmony/drlvm/build/make/build_component.xml:72: The following 
error occurred whi le executing this line:
/home/robing/harmony/drlvm/build/lnx_ia32_gcc_debug/semis/build/targets/build.native.xml:75:
 g cc failed with return code 1


AFAIR, HARMONY-443 solved this problem. I am afraid, not all patches
were applied from the attachments. Though, you can find many solutions
there

BTW, I would vote for enabling gcc 4.x build ASAP


+1

I've now built successfully with gcc 3.4, which serves my purposes for now.

Thanks to everyone for the assistance.

cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] Build problems - Stl.h

[Robin Garner]

Geir Magnusson Jr. wrote:


On Sep 22, 2006, at 12:02 AM, Robin Garner wrote:


Geir Magnusson Jr. wrote:


IIRC, there's something broken about us and gcc 4.
Go back to gcc 3.4.6 or so...
geir


OK, so I worked out I do this by editing lnx.properties and changing

 # Default C compiler, either icc or gcc
-CXX=gcc
+CXX=gcc-3.4


Ew...  c'mon

$ ln -s /usr/bin/gcc-3.4  gcc
$ ln -s /usr/bin/g++-3.4 g++


Eurghh.  Yes I know this would do, but I was kind of hoping for 
something resembling elegant :)







common_extra:
BUILD FAILED
/home/robing/harmony/drlvm/build/make/build.xml:404: The following 
error occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build.xml:411: The following 
error occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build_component.xml:72: The 
following error occurred while executing this line:
/home/robing/harmony/drlvm/build/lnx_ia32_gcc-3.4_debug/semis/build/targets/common_extra.xml:8: 
gcc-3.4 is not a legal value for this attribute


what next ?  :)



See above :)  Hopefully you are getting the idea that our build for 
DRLVM is very fragile (and need to be replaced).  Don't taunt the build


geir


Thanks,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] Build problems - Stl.h

[Robin Garner]

Geir Magnusson Jr. wrote:



IIRC, there's something broken about us and gcc 4.

Go back to gcc 3.4.6 or so...

geir


OK, so I worked out I do this by editing lnx.properties and changing

 # Default C compiler, either icc or gcc
-CXX=gcc
+CXX=gcc-3.4


common_extra:

BUILD FAILED
/home/robing/harmony/drlvm/build/make/build.xml:404: The following error 
occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build.xml:411: The following error 
occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build_component.xml:72: The following 
error occurred while executing this line:
/home/robing/harmony/drlvm/build/lnx_ia32_gcc-3.4_debug/semis/build/targets/common_extra.xml:8:
 gcc-3.4 is not a legal value for this attribute



what next ?  :)

Thanks,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] Build problems - missing bfd.h

[Robin Garner]

Geir Magnusson Jr. wrote:


On Sep 21, 2006, at 9:18 PM, Robin Garner wrote:


Geir Magnusson Jr. wrote:

On Sep 21, 2006, at 1:09 AM, Robin Garner wrote:

Trying to build the latest trunk (r448461) on Ubuntu (6.06).

That's my main build platform.  No need to hack anything.

  Had to hack build.sh because the ant executable isn't in ANT_HOME,

That's weird.  You shouldn't have to do that.


These fragments from build.sh:

# Apache Ant 1.6.2 or higher (can be obtained at http://ant.apache.org)
if [ -z "$ANT_HOME" ]; then
ANT_HOME=$EXTERNAL_DIR/common/apache-ant-1.6.5-bin/apache-ant-1.6.5
fi
...
elif [ ! -x $ANT_HOME/bin/ant ]; then
   echo "* File $ANT_HOME/bin/ant not found."
   echo "* Make sure you have Ant 1.6.5 or above installed from"
   echo "* http://ant.apache.org/bindownload.cgi and that ANT_HOME 
environment"

   echo "* variable points out to the Ant installation dir, e.g. "
   echo "* export ANT_HOME=/usr/local/ant_1.6.5"
   ERROR
fi
...
export ANT_HOME=`(cd $ANT_HOME;pwd)`
...
ANT_COMMAND="$ANT_HOME/bin/ant --noconfig"

seems to indicate otherwise, if you want to use the default ubuntu ant 
install.  Doing "export ANT_HOME=`(cd $ANT_HOME;pwd)`" seems a bit 
*cough* 'suboptimal' :)


I don't know about the default ubuntu ant install.  I installed it 
myself and set ANT_HOME to the right place in .bashrc.  I never futz 
with it...


Still, if I have a working ant in my PATH, the script should just 
respect that, shouldn't it ?



-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[DRLVM] Build problems - Stl.h

[Robin Garner]

Thanks to everyone who assisted before ... the build is progressing, but 
now fails with



   [cc] /usr/lib/gcc/i486-linux-gnu/4.0.3/../../../../include/c++/4.0.3/bits/stl_tree.h:36 7: 
error: no matching function for call to ‘Jitrino::StlMMAllocator::construct 
(std::pair*, const std::pair&)’
   [cc] /home/robing/harmony/drlvm/vm/jitrino/src/shared/Stl.h:123: note: candidates 
are: void Jitrino::StlMMAllocator::construct(T*, const T&) [with T = 
Jitrino::Opnd*]

BUILD FAILED
/home/robing/harmony/drlvm/build/make/build.xml:404: The following error 
occurred while execut ing this line:
/home/robing/harmony/drlvm/build/make/build.xml:411: The following error 
occurred while execut ing this line:
/home/robing/harmony/drlvm/build/make/build_component.xml:72: The following 
error occurred whi le executing this line:
/home/robing/harmony/drlvm/build/lnx_ia32_gcc_debug/semis/build/targets/build.native.xml:75:
 g cc failed with return code 1



Thanks in advance,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] Build problems - missing bfd.h

[Robin Garner]

Geir Magnusson Jr. wrote:


On Sep 21, 2006, at 1:09 AM, Robin Garner wrote:


Trying to build the latest trunk (r448461) on Ubuntu (6.06).



That's my main build platform.  No need to hack anything.


  Had to hack build.sh because the ant executable isn't in ANT_HOME,


That's weird.  You shouldn't have to do that.


These fragments from build.sh:

# Apache Ant 1.6.2 or higher (can be obtained at http://ant.apache.org)
if [ -z "$ANT_HOME" ]; then
ANT_HOME=$EXTERNAL_DIR/common/apache-ant-1.6.5-bin/apache-ant-1.6.5
fi
...
elif [ ! -x $ANT_HOME/bin/ant ]; then
   echo "* File $ANT_HOME/bin/ant not found."
   echo "* Make sure you have Ant 1.6.5 or above installed from"
   echo "* http://ant.apache.org/bindownload.cgi and that ANT_HOME 
environment"

   echo "* variable points out to the Ant installation dir, e.g. "
   echo "* export ANT_HOME=/usr/local/ant_1.6.5"
   ERROR
fi
...
export ANT_HOME=`(cd $ANT_HOME;pwd)`
...
ANT_COMMAND="$ANT_HOME/bin/ant --noconfig"

seems to indicate otherwise, if you want to use the default ubuntu ant 
install.  Doing "export ANT_HOME=`(cd $ANT_HOME;pwd)`" seems a bit 
*cough* 'suboptimal' :)



and add a URL to download xalan


No need to do that either.  The xalan problem means that DRLVM wasn't 
finding the classlib tree correctly, where the Xalan dep is.   By 
default, DRLVM assumes that classlib and it are arranged on the 
filesystem as they are in SVN :


   enhanced/
 classlib/
 trunk/
 drlvm/
trunk/

but if you are using the "federated build" that is enhanced/trunk, then 
the relationship is different.


The best way is to look at   build/drlvm.properties.example, copy to 
build/drlvm.properties and adjust so it's right.


Ah ... yep, works a treat.

, but now the build fails with a missing "bfd.h" (full error message 
below).


you need to install binutils for bfd, IIRC.


binutils-dev does the trick.

Thanks,
Robin



Any pointers ?

cheers,
Robin

--

build.native.c:
   [cc] Starting dependency analysis for 10 files.
   [cc] 10 files are up to date.
   [cc] 0 files to be recompiled from dependency analysis.
   [cc] 2 total files to be compiled.
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:21:17: 
error: bfd.h: No such file or directory
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:22:21: 
error: dis-asm.h: No such file or directory
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:63: 
error: syntax error before ‘fprintf_ftype’

   [cc] cc1: warnings being treated as errors
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:65: 
warning: ‘struct disassemble_info’ declared inside parameter list
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:65: 
warning: its scope is only this definition or declaration, which is 
probably not what you want
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:70: 
error: field ‘bfd_info’ has incomplete type
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: 
error: syntax error before ‘bfd_decoder’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: 
warning: initialization makes integer from pointer without a cast
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: 
warning: data definition has no type or storage class
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:89: 
error: syntax error before ‘src’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘disasm_read_memory’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: 
error: ‘buffer’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: 
error: (Each undeclared identifier is reported only once
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: 
error: for each function it appears in.)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: 
error: ‘src’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: 
error: ‘n’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: At top 
level:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:103: 
error: syntax error before ‘memaddr’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘disasm_print_adress_default’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:105: 
error: ‘info’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/dr

Re: [drlvm][jit] MMTk-style magics implementation in Jitrino.OPT compiler

[Robin Garner]

> On 9/21/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
>>
>> Weldon, Robin
>> thank you for the comments.
>> There are just a few steps left to do before Jitrino.OPT will have
>> usable
>> "unboxed" package implementation.
>>
>> Right now I'm working on atomic operations (prepare/attempt pair), and I
>> do
>> not completely understand the semantic of the 'prepare' method.
>> The "prepareXYZ" method looks like a simple load and is not an atomic
>> operation by itself. Are there any examples that describe these
>> operations
>> in details?
>
>
>
> Robin will correct me if I am wrong.
>
> I think the idea is that this is a generic API intended to be used for
> accessing "compare and swap" facilities on a bunch of different hardware.
> And the actual hardware has different memory models, different CAS
> semantics, etc.
>
> Basically the generic facility needed is to first read the CAS target
> memory
> location (this is the "prepare").  If the memory is sitting at a specific
> value (this is the "do something"), then do an "attempt" as follows:
>
> old_contents =  CAS(expected_specfic_value_that_I_just_read,
> some_new_value,
> target_memory_location)
>
> If the old_contents are identical to
> expected_specific_value_that_I_just_read, we know that the CAS succeeded.
> In other words, you won the competition.  Otherwise you lost.
>
> I imagine the following situation in user's code:
>>
>> prepareXYZ()
>> do something
>> attempt()
>>
>> Is this code correct?
>>

You're correct.

The prepare/attempt was designed to match architectures (PPC, Alpha ...)
with a load-locked/store-conditional instruction pair.

The standard code fragment for a synchronized update is

do {
  oldVal = x.prepare();
  newVal = f(oldVal);
} while (!x.attempt(oldVal,newVal));

As an aside, vmmagic should probably have CAS as a primitive rather than
prepare/attempt, but I don't think that's going to happen in the immediate
future.

cheers


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[DRLVM] Build problems - missing bfd.h

[Robin Garner]

Trying to build the latest trunk (r448461) on Ubuntu (6.06).  Had to 
hack build.sh because the ant executable isn't in ANT_HOME, and add a 
URL to download xalan, but now the build fails with a missing "bfd.h" 
(full error message below).


Any pointers ?

cheers,
Robin

--

build.native.c:
   [cc] Starting dependency analysis for 10 files.
   [cc] 10 files are up to date.
   [cc] 0 files to be recompiled from dependency analysis.
   [cc] 2 total files to be compiled.
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:21:17: 
error: bfd.h: No such file or directory
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:22:21: 
error: dis-asm.h: No such file or directory
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:63: error: 
syntax error before ‘fprintf_ftype’

   [cc] cc1: warnings being treated as errors
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:65: 
warning: ‘struct disassemble_info’ declared inside parameter list
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:65: 
warning: its scope is only this definition or declaration, which is 
probably not what you want
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:70: error: 
field ‘bfd_info’ has incomplete type
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: error: 
syntax error before ‘bfd_decoder’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: 
warning: initialization makes integer from pointer without a cast
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:81: 
warning: data definition has no type or storage class
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:89: error: 
syntax error before ‘src’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘disasm_read_memory’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: error: 
‘buffer’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: error: 
(Each undeclared identifier is reported only once
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: error: 
for each function it appears in.)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: error: 
‘src’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:94: error: 
‘n’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: At top level:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:103: error: 
syntax error before ‘memaddr’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘disasm_print_adress_default’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:105: error: 
‘info’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:105: error: 
‘memaddr’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘disasm_print’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:177: error: 
‘bfd_vma’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:177: error: 
syntax error before ‘apr_uint32_t’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:183: error: 
syntax error before ‘apr_uint32_t’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘port_disasm_initialize’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:297: error: 
‘disassembler_ftype’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:297: error: 
syntax error before ‘bfd_print_insn_sym’
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c: In 
function ‘port_disassembler_create’:
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:330: error: 
‘bfd_arch_i386’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:331: error: 
‘bfd_mach_i386_i386_intel_syntax’ undeclared (first use in this function)
   [cc] 
/home/robing/harmony/drlvm/vm/port/src/disasm/linux/disasm.c:339: error: 
‘BFD_ENDIAN_LITTLE’ undeclared (first use in this function)


BUILD FAILED
/home/robing/harmony/drlvm/build/make/build.xml:404: The following error 
occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build.xml:411: The following error 
occurred while executing this line:
/home/robing/harmony/drlvm/build/make/build_component.xml:72: The 
following error 

Re: [drlvm][jit] MMTk-style magics implementation in Jitrino.OPT compiler

[Robin Garner]

Weldon Washburn wrote:

On 9/20/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:


I put the update into JIRA. This update still has a lot of bugs, but is
significantly more stable then the initial one.
My plan is to work on lock prefix support in Jitrino.OPT CG  tomorrow, so
if
somebody is interested to enhance the current implementation there 
will no

conflicts in our work.

+ There are several issues to discuss that are not clear to me:

1) Do we allow to save magic class into object fields? If yes, GC must be
aware about magics and do not enumerate magic fields.



Actually, the GC issues are taken care of at a higher level.  The developer
who is using vmmagics must extend the "Uninterruptible" class.  


That's not actually strictly true, there's also the 
UninterruptiblePragma exception.  More generally, the burden is on the 
programmer to ensure that objects that you hold an Address of don't move 
while you hold the reference, and these pragmas are one method of doing 
that.  You could also use locks etc (in theory at least).



  Another
issue is that storing an unboxed object from vmmagic into an arbitrary
object field has to be thought through very carefully by the developer.  Or
else there will be stale pointers running around that will cause hard
crashes.  In any case it is not a JIT responsibility.  If the developer 
puts

an object of type vmmagic Address in the heap then later retrieves a stale
pointer, disaster will hit.  I know because I have done this accidentally.


Magic loads and stores also don't trigger read/write barriers, so the 
programmer needs to manually insert barriers if they would have been 
called.



4) Why do we need all of these types: Word, Offset, Extent if they are all

just platform dependent unsigned integers? I understand that code of
garbage
collectors already uses these types and we must support them all, but 
what

was the initial reason to introduce all of them?



I thought the same thing when I first started working on MMTk port.  Now I
see there is benefit in the different types because it makes the code 
easier
to read and also allows java type system to catch dumb errors that 
otherwise

would be left to debugging stage.


Once upon a time, magic fields were all just ints, and the present 
variety evolved over time, precisely for this reason.


An ObjectReference may seem mysterious to the casual observer (why don't 
we just use Object ?).  This is present so that MMTk can support 
languages other than Java (Haskell was my original target, and we have 
done C#), and also so because the object model of the VM that executes 
MMTk code may not necessarily be the same as the system that MMTk is 
managing.


So an ObjectReference is the 'moral equivalent' of an object.


Thats all for today. Will provide an update in a day or two.




On 9/18/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
>
> On 9/18/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
> >
> > All,
> > I'm working on the implementation of MMTk's
> > "org.vmmagic.unboxed"
> > package functionality in Jitrino.OPT compiler.
> > If you are interested to participate in the development, I propose to
> > discuss all details in this mail thread.
> >
> > The current state:
> > Part of the functionality of vmmagic package is done in the
magic1.patch
> .
> > See JIRA 1489 (http://issues.apache.org/jira/browse/HARMONY-1489)
> >
> > Tasks that are not finished:
> > 1) Support of unsigned types.
> > 2) Support of atomic prepare/attempt operations
> > 3) Testing suit for vmmagic package.
> > 4) EM64T support
> >
> >
> > I hope items 1) and 2) will be finished in a week or even sooner if
> > someone
> > helps. After it's done the item 4) won't be a problem.
> > I think that the problem (at least for me) is item 3): we need a test
> > suite
> > for vmmagic package. I saw several tests in Weldon's
drlvm/trunk/vm/MMTk
> > folder, but this is not sufficient to cover the whole vmmagic 
package.

> > Does
> > anyone know/can_write a reliability test suite for vmmagic we can use
in
> > Harmony?
>
>
>
> A regression test for vmmagic exists.  I have been trying to get it
> donated
> and posted to MMTk repository.  Its been a couple of months and no
> response.  I will find out if it can be donated to Apache.
>
> --
> > Mikhail Fursov
> >
> >
>
>
> --
> Weldon Washburn
> Intel Middleware Products Division
>
>


--
Mikhail Fursov








-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

I don't understand. How can weak references help short-lived objects 
reclaim?


Really what I'm saying is that this is the closest thing we have to a
hint to GC that objects can be collected soon - but it is not anything
like a proper free() call. There is no immediate reclaim of memory,
just the possibility that it will be reclaimed soon - and the object
may be garbage collected before you are finished with it!

Regards,
Oliver



Actually, it's kind of the other way around isn't it ?  Nulling the last 
pointer to an object tells the GC that it can collect it (explicitly in 
the case of reference counting), whereas having a Weak Reference to an 
object says 'please tell me when on-one else wants this object', which 
results in the object staying around even longer.


cheers,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM][JIT] can Jitrino.JET call MMTk alloc() instead of C helper?

[Robin Garner]

Weldon Washburn wrote:

On 9/8/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:


Weldon,
I added 'alloc' support to JIRA 816 (magic2.zip file)
Now it supports objects allocation only, not arrays. MMTk allocation 
works
only for class named 'test'  and methods with prefix  'testAlloc'. To 
turn

it on use "-Xjit jet::alloc4j" option.



Thanks!  I will take a look this weekend.

I had a problem with implementation: usual 'new' vm helper does vtable 
field

initialization in allocated object space by itself. Your alloc() method
doesn't.
In the patch in JIRA I added vtable initialization code right after the
alloc() call to JIT, that is actually bad design decision (IMO).



You are correct.  Its bad design.  Writing the vtable pointer is not the
responsibility of the JIT.  MMTk is somewhat vague on who owns vtable ptr
init.  In any case, it looks like the jit should follow the call to alloc()
with a call to postAlloc().  I will make sure the postAlloc() code fixes up
the vtable ptr.  Its OK to "bend" the typeRef.  All I want is the hard ptr
that gets written into the object header.  We will go back later and fix
this interface hack.  See below:


Ummm ... MMTk isn't vague on who owns vtable ptr initialization - it's 
the responsibility of the VM.  MMTk strives to know as little about the 
object model as possible, apart from what is exposed through the 
ObjectModel interface.  OK, perhaps the contract isn't as clearly 
expressed as it chould be :)


In JikesRVM, the incoming interface (MM_Interface) provides the 
vm-specific features such as adding the object header size to the 
allocation request, filling in the non-GC portions of the header etc.  I 
think that approach would work for DRLVM.


MMTk definitely doesn't want to know about vtables and type pointers - 
after all there are plenty of systems it could be built into that don't 
have such things.




/**

* Perform post-allocation actions. For many allocators none are

* required.

*

* @param ref The newly allocated object

* @param typeRef the type reference for the instance being created

* @param bytes The size of the space to be allocated (in bytes)

* @param allocator The allocator number to be used for this allocation

*/

public void postAlloc(ObjectReference ref, ObjectReference typeRef,

int bytes, int allocator) throws InlinePragma {





Can we replace the 'alloc()' call with another Java method  that uses

unboxed API and behaves exactly like 'new' helper?


I think that's what I'm saying MMTk expects.

cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

Weldon Washburn wrote:


Its not a simple wrapper that is missing.  MMTk is written in Java.  This
Java code needs to be intergrated into the bootstrap process of DRLVM.   
For

example, initial bootstrap java code needs to run on a bootstrap java heap
until all of MMTk itself has been compiled and initialized.   I keep hoping
that the MMTk guys will volunteer to help me with this (hint, hint).  
Having
said that, I am trying to put together a TODO list of stuff that needs 
doing

on the MMTk port so that others can jump in and help.  I hope to have this
list done within one week.



Happy to help if you have specific questions :)

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

Egor Pasko wrote:

On the 0x1E4 day of Apache Harmony Oliver Deakin wrote:

Forcing gc by hand does work, but it is difficult for code to know
when to
call gc.So I think it is better if VM can give some support since it
knows
the global situation.:)

..and of course a manual gc() call does not necessarily result in a gc


this one is easy :) make a gc2() magic in DRLVM and collect only young
objects on it.. to make code portable, implement gc2() as gc() for
other JVMs. Need to be patient until the generational GC.

Can a hack like this be widely accepted in Java community? I think, yes.



I think GC research will soon make this unnecessary.  For example the 
Sun VM has a concurrent collector for its mature space, so on a 
sufficiently lightly loaded system (or with a core to spare) you could 
eliminate most of the mature space collection pause time.


In MMTk, there is a command line switch, -X:gc:fullHeapSystemGc, used to 
specify whether System.gc() causes a full heap collection.  But no, I 
don't think this is something users should need to play with.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

   What about the VMs here, drlvm or J9?

In a DRLVM JIT (Jitrino.OPT) there is an escape analysis prototype. It
detects objects that can be allocated on stack (and, hence, on
registers). Currently, it is switched off by default, and, when
enabled, it just marks the objects that are not escaped. This info is
never used in Jitrino.OPT yet. Sometimes, escape analyzers help, but
not very much :)



It will be great if JIT can help to allocation objects on stack.

DRLVM GC gurus will say more ;)


The JikesRVM experience was that stack allocation didn't improve 
anything, but once you can show that an object never escapes a method, 
and is 'hot', you can actually inline the fields of the object, and 
never allocate it in the first place.  This is a valuable optimization.


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

Egor Pasko wrote:

On the 0x1E4 day of Apache Harmony Alexey Varlamov wrote:

Just a wild idea: a smart JIT could hint a GC during allocation if an
object is expected to be short-lived so the GC could allocate it in a
special space, 


if a JIT can prove that the object is local, it can allocate it on
stack almost without help of VM. An imprecise estimation of
short-liveness is a kind of magic. What heuristics can we use? Small
objects live shortly? :)



Stack allocation generally performs more poorly than heap allocation.

In fact the cheapest place to allocate a short-lived object is in the 
nursery where all objects are allocated - the cost of allocation is 
simply incrementing a counter and doing a bounds check.


Identifying objects (usually call sites) that are likely to live for a 
long time, and allocating them directly into the mature space is called 
pre-tenuring, and there has been a lot of work done on that.


ALl this can be done by the JIT at run-time, although feedback from the 
VM via sampling can be useful.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [General VM] GC strategy:how to garbage collect short-lived objects quickly.

[Robin Garner]

Designing a garbage collector with low pause times and high throughput (ie
low overhead) is to an extent the 'holy grail' of memory management
research.

The worst case pause times are in full heap collectors, where the pause
time is proportional to the total number of live objects in the heap.

Generational collection helps drive down the average pause time (nursery
collections are cheap) but the maximum pause time (when the mature space
needs collecting) is much the same as a full heap collector.  A
generational collector with a bounded nursery can shorten the time of
minor collections, but has lower throughput and the same maximum pause
time.

To an extent, most of the literature on GC addresses this issue, and some
things you might like to read up on are:

- Metronome.
  This pushes pause times down into the sub-millisecond realm, but at a
considerable (although predictable) overhead.

- The Sun 1.5 JVM, which has an optional concurrent collector for the
mature space

- Ulterior Reference Counting, which combines the high throughput of a
generational collector with the low pause times of reference counting.

- Beltway, which collects the heap in small chunks, after giving objects
time to die

- Free-me, which automatically identifies short-lived objects

Just to name a few.

cheers

> Leo Li wrote:
>> Hi,all:
>>As we all know, java objects are allocated on heap instead of stack,
>> thus there is a problem about how to garbage collect short-lived objects
>> quickly.
>>In a recent real project I involved, a server built on java tries to
>> send thousands of messages to client per second. A lot of short-lived
>> messages is created as objects and discarded. (Although I can recycle
>> these
>> memory, there is still a byte array created during per call of nio read
>> and
>> write.) Since current GC strategy adopted by current RI starts to work
>> only
>> when the memory allocated approaching the limit of java heap, the work
>> of GC
>> is huge and will raise a wave on the server performance. Furthermore,
>> after
>> a long run, although I know GC will merge memory, the operating system
>> reports there is memory fragment and in the worst case the OS will even
>> report real memory is exhausted.
>>Of course it is possible to limit the java heap so as to force gc
>> frequently as a workround, is it preferrable to collect short-lived
>> objects
>> quickly such as adopt aged-related object queues as one of the gc
>> strategy?
>>   What about the VMs here, drlvm or J9?
>>
>
> Interesting topic, I'm still dreaming of "free()" in Java (This dream
> begins at the very beginning when I see Java, as C/C++ is my first
> program language )However, it seems RI will never give us "free()". :)
>
> Only a thought, Java may offer a key word "temp", indicating that this
> variety can be freed at once.
>
>> Leo Li
>> China Software Development Lab, IBM
>>
>
>
> --
>
> Best Regards!
>
> Jimmy, Jing Lv
> China Software Development Lab, IBM
>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>



-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] GC heap verification infrastructure

[Robin Garner]

Weldon Washburn wrote:

On 7/18/06, Volynets, Vera <[EMAIL PROTECTED]> wrote:

Hi,
I didn't find any bugs in gc_v4 at the moment,


Vera, its good to know the verifier found zero bugs in GCV4!


but this feature is really very useful especially if
you modify gc_v4 or design a new gc. Of course, it needs to
be improved and developed.


This brings up a good point.  Harmony-dev needs to discuss the roadmap
for DRLVM GC.  I will start a discussion thread shortly.



Also I'm going to add feature to verify write barriers work.


Some comments.

1)
There is no DRLVM GC that uses a write barrier currently.  Assuming
you have a write barrier verifier, how will you know it works?

2)
I would like to see the design of the write barrier verifier discussed
on harmony-dev mailing list before implementation is started.  I am
interested in using the proposed write verifier on DRLVM/MMTk.


The first method to occur to me (and I'm afraid this is shamelessly 
MMTk-centric :) is to write a write-barrier validating plan:


- Mutator runs, using write barrier to populate the REMSET
- Collector performs a full-heap collection (a-la IGNORE_REMSETS), 
however, whenever an old-to-new pointer is encountered, you check 
whether the pointer was recorded in the REMSET.

- Ditto for roots that point into the nursery.

It would need to take care not to flag references from objects promoted 
from the nursery earlier in this collection, but this shouldn't be too 
hard to arrange.


This should also be quite easy to build entirely within MMTk (sorry GCV5 
guys :)


Another (better) approach would be to write a REMSET sanity checker. 
When enabled by a command-line switch it would inject an extra phase 
into the GC phase list (like the existing sanity checker does), and 
before a nursery collection it would:


- do a full-heap trace, using private side metadata
- Record all references that point into the nursery
- when done, check this against the remset and report discrepancies.

The remset should contain a superset of the references that this sanity 
checker finds.  There's no real reason this couldn't live in MMTk 
permanently.



The same way as in GC heap verification I use idea to
hijack interface function concerning write barriers.

This patch will be sent a bit later.

Pleased to hear from you, Vera :)!

-Original Message-
From: news [mailto:[EMAIL PROTECTED] On Behalf Of Egor Pasko
Sent: Saturday, July 15, 2006 9:37 AM
To: harmony-dev@incubator.apache.org
Subject: Re: [DRLVM] GC heap verification infrastructure

On the 0x1A6 day of Apache Harmony Vera Volynets wrote:
> *   GC heap verification infrastructure   ***
> Hi,
> I have been working on implementing GC heap verification
infrastructure
> for Stop-The-World GC in DRLVM.
> This infrastructure is intended be used with any stop-the-world GC
> implementation, conforming to the GC-VM interface (described in gc.h
and
> vm_gc.h), with only a minor GC-VM interface change.
> It works on Windows and Linux ia32 platforms.

cool feature! did you find any bugs in DRLVM while testing?
do you have plans/ideas for future development of the tool?

> [...]
>
> *Patch contents*
> This is an individual contribution, and it was prepared according to
the
> requirements of Apache cleanroom process; the size of two files all in
> all is about 30kb (161 and 585 lines).
> -Patch Add-function-to-gc-interface.txt

oh, I found it! HARMONY-881

--
Egor Pasko, Intel Managed Runtime Division


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]








-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM][VM] -- which header bits are available for GC mark and GC forwarding use?

[Robin Garner]

Weldon Washburn wrote:

Robin,

Good points.  Given that Object.hashCode() implementation sortof,
kindof depends on a copying mature space, does it make sense for the
GC to "own" the Object.hashCode() implementation?  That way, we
eliminate the vm-wide debate about giving object hash one or two or
even 12 header bits.  


Absolutely.  +1

>  It all becomes a local problem of GC

implementation.  No need for negotiating with the rest of the VM :)
 From a top-level how about giving the GC one byte of object header to
do hash plus whatever it wants?  I realize it would be nice to have an
"expando" object model in Harmony that would allow arbitrary number of
header bits for a) tib ptr, b) default hash code, c) lock info and d)
GC info.  I worry that this would be too disruptive to the code base
right now.  Can we make do with the above proposal?


We can definitely make do with it.



Just to be clear, the two-bit Object.hashCode() scheme I refer to is
the one in section 3.3 of, "Space- and Time-Efficient Implemention of
the Java Object Model" by David Bacon, et al.  I think this is also
the scheme Ivan Volosyuk is refering to and has an implementation of.


And as implemented in JikesRVM.  Yep, that's the one I'm talking about 
too.  So where you say


>> >   I think the hashCode can be
>> > reduced to one bit plus the object's current address at first
>> > HashCode() invocation.  I'd put this hash bit in the GC byte.  And
>> > make the GC byte the lowest byte in the header word.  The remaining
>> > 3bytes could be used for fat/thin locks.

I'm guessing you're assuming a copying nursery and a non-moving mature 
space ?  In a more general setting I'm not sure how you could get away 
with one bit.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM][VM] -- which header bits are available for GC mark and GC forwarding use?

[Robin Garner]

>  From talking to
> the MMTk guys (Steve Blackburn) it seems MMTk wants to have one byte
> of object header for private use.  Its unclear to me if this will be a
> performance problem for a product JVM.

Yep.  We found a nice mark-sweep implementation technique that greatly
reduces the cost of sweeping if we have several mark bits available, but
after 4 or so bits the advantage tails off, so there is room for
compromise with the VM's need.  On the other hand I'm currently looking at
other ways we could speed up GC if there were a few extra header bits
available, so the more bits available to GC, the better - and these will
generally be tunable parameters that can be traded against the VM's other
needs.

>   I think the hashCode can be
> reduced to one bit plus the object's current address at first
> HashCode() invocation.  I'd put this hash bit in the GC byte.  And
> make the GC byte the lowest byte in the header word.  The remaining
> 3bytes could be used for fat/thin locks.

In GenMS, where there is a copying nursery and a non-moving mature space,
you could indeed get away with 1 bit for hashcode, with a different
meeaning in each space.  In a heap where objects could move more than
once, I think you might still need 2 bits.

cheers


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm][vm, mmtk] adding thread-local storage functionality to vmmagic

[Robin Garner]

Egor Pasko wrote:

On the 0x1C9 day of Apache Harmony Mikhail Fursov wrote:
  

Would id be possible to add the Java TLS API into VMThreadManager like:


long  tlsAlloc(); // returns a key for new local storage entry
tlsSet(long key, long dataAddr); // stores a pointer to memory addr
for the given key
long tlsGet(long key); // returns memory addr for the given key
void tlsFree(long key); //  releases the given entry in TLS
?
  



 
  

The benefit of that approach could be that fs14 knowledge is localized
within the threading module and is not exposed to any other VM
component like GC or JIT.

  

Once JIT is responsible to generate TLS access magic, that is platform
dependent by itself, I do not understand how you can hide this knowlegde
from JIT. If  vmmagic uses abstract TLS access operation like described
above but not FS[14] it's impossible to teach JIT to generate FS[14] access
using vmmagic :)



As far as I understand, Andrey proposes to have a number of low-level
JIT vmmagics (like .. fs[14], etc.) and write high-level accessors
like tlsSet, tlsGet using these magics. Andrey says, it helps to
localize low-level vmmagic usage in one place. JIT<->threading tight
dependence persists here in form of low-level accessors.

But.. I see one major disadvantage here. You should write your tlsSet
and tlsGet in Java and in a system-dependent way. This is what I think
we should avoid.

The alternative method is to allow tlsSet and tlsGet be vmmagic
themselves. Now system-dependent code is hidden within a JIT. The
JIT<->Threading is probably more tight here because JIT takes more
code as vmmagic. Another obvious disadvantage is that JIT has to do
more here. 
  

This is a slightly less general approach than what I suggested here:

> My thoughts on the thread-local storage issue is that it's a more 
general issue of accessing > shared data structures from the Java side 
of the fence.  So perhaps a method such as

>
> Object org.vmmagic.utility.accessVmStructure()

where the identifier is some abstract constant that identifies all of 
the vm-specific data structures one might want to access.  Some of them 
might be slots in the tls area, but others might be elsewhere in the VM.


As long as vmmagic is processed by the JIT after some constant 
propagation takes place it should be quite possible to compile this down 
to an optimal sequence of instructions.

Hm, but I like this way more. The idea is quite elegant: hide
system-dependent implementation details in high-level magics and let
JIT optimize them out. Provide a slow implementation in API in case
JIT does not recognize the magic. I Hope, there will be not a lot of
large high-level system-dependente vmmagics around. In this case this
approach is ideal for me.

  
I agree completely, and heartily recommend using a small number of 
high-level features for these things.  When I first started work porting 
MMTk, the interface to the VM had a large number of ad-hoc methods for 
accessing all sorts of things.  The present state of vmmagic reflects a 
lot of work and thought by quite a few people on how to abstract over 
the many low-level mechanisms that the VM and MMTk used to use to interact.


The idea of providing a slow implementation of higher level vmmagic in 
Java is interesting, but begs the question of which vmmagics you are 
ultimately going to use to implement *them*.


Finding the right level of abstraction has a payoff on both sides of the 
fence - the java code is cleaner, and there should be less work on the 
JIT side to implement a single general-purpose mechanism.




-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm][vm, mmtk] adding thread-local storage functionality to vmmagic

[Robin Garner]

> On 8/18/06, Robin Garner <[EMAIL PROTECTED]> wrote:
>>
>> Object org.vmmagic.utility.accessVmStructure()
>
> This is a good idea to keep in mind. We can introduce it when needed.
> At the moment for service fast-path inlining, I guess specific JIT
> intrisics are enough.

It strikes me that this single intrinsic is extensible and just as easily
implemented as the JIT specific ones.  And heads off a potentially messy
proliferation of ad-hoc methods right at the start.

>> or even better, declare object instances with a pragma annotation
>>
>> @IntrinsicDataStructure("name")
>> MutatorContext myMutatorContext;
>>
>> and the compiler generates code to fetch the address.  Has better type
>> safety, potentially.
>
> hmm, I am not sure how this syntactic suger can have better type safety.

Because an annotation that is seen by the JIT gives it the opportunity to
check the type of the declared instance with the VM structure it
corresponds to.  I think dismissing annotations as syntactic sugar is a
bit unfair.

> Thanks,
> xiaofeng
>

cheers,
Robin


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm][vm, mmtk] adding thread-local storage functionality to vmmagic

[Robin Garner]

Egor Pasko wrote:

On the 0x1C8 day of Apache Harmony Mikhail Fursov wrote:
  

Egor,
Why do you think we need to support platform specific issues in helpers? Is
there any example that shows that we can't write a helper without reference
platform details? 



It depends on how much of a helper code we want to write in Java. We
can always say: "hey, this is platform-dependent, let it be a whole
vmmagic and let JIT take care". If it is a large piece of code,
it is not good to code it down in JIT IR(s) and is better written in
Java. I think, we all agree here.

As a 1 more example ... maybe some day we decide to inline fast-path
of spinlocks. 

Which is a) a helper b) is platfrom-dependent. 


Implementing fast-path-spins in JIT IR is not a lot to do. But
specifying the interface and updating all components (JITs) to handle
that way of interaction on all platforms .. may make a burden. Coding
spin-locking in Java would make it difficult to write optimal (think of
explicitly parallel architectures...)

Both approaches seem to be quite complicated here. But all-in-JIT
approach seems to have more chances to be faster here.

Thus, it would be good to have both:
* vmmagic (inlined by JITs)
* helpers (partially) written in Java (using vmmagic)
and decide how fine-grained those magics should be depending on
specific situation.

  

I think we should avoid #ifdef's in Java as much as we can.



Yes, my vision is the same. At least, until we come accross a large
piece of system-dependent code that is good to be inlined by JIT. I
think, it is not likely to happen, large pieces of code tend to work
longer :) and inlining them does not give so much benefit..

  
Jikesrvm uses "//-#if" style constructs for platform-specific codes, and 
it's something we are working hard to get rid of.  I think it's 
something you should definitely avoid at all costs.


One possible alternative is to use annotations instead, and the 
preprocessing infrastructure that goes with them.  But it's always 
better to try to abstract over the particular concern somehow.


As a last resort, write platform-specific versions of whole classes, and 
selectively include them at compile time.  Of course chase down the 
platform specific bits so they are as small as possible.


My thoughts on the thread-local storage issue is that it's a more 
general issue of accessing shared data structures from the Java side of 
the fence.  So perhaps a method such as


Object org.vmmagic.utility.accessVmStructure()

or even better, declare object instances with a pragma annotation

@IntrinsicDataStructure("name")
MutatorContext myMutatorContext;

and the compiler generates code to fetch the address.  Has better type 
safety, potentially.


-- robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM][GC] Goals for 2006/2007

[Robin Garner]

Weldon Washburn wrote:

All,

There is rough consensus that the immediate goal for Harmony JVM is to
reliably run simple commercial workloads with acceptable performance.
In regards to a garbage collector for a Harmony JVM in 2006 there are
some data points worth noting.  1) A quick survey shows most basic
commercial JVMs implement a generational collector. 2) While the
existing drlvm garbage collector, gcv4, implements some interesting
advanced concepts, it is not currently a generational collector.  3)
The MMTk port to drlvm is not yet finished.  Even assuming MMTk's
generational configuration is appropriate, it is still too early to
put this garbage collector on the roadmap for a 2006 Harmony JVM.  It
might be worth revisiting in 2007.  But it's too far away to debate at
this time.

Given the above data points, the following is a first stab at
requirements for Harmony "GCV5".  The intention is to set down some
basic parameters.

1)
Generational Collector with mark/compacting mature object space
Why mark/compact specifically ?  The easiest approach would be to add a 
copying nursery 'in front' of the exiting GCV4 mature space, and then 
look at replacing/updating the implementation of the mature space.  This 
could be achieved with virtually no change to the mature space collector.


As an aside, best performance with a generational collector also comes 
from an Appel-style nursery, ie the nursery size is essentially 
(heap-mature)/2.


The rest of the worklist seems uncontroversial to me, but I wonder how 
much work it is to implement these vs getting MMTk working.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: performance of drlvm

[Robin Garner]

There was a paper last year at OOPSLA about similar things, Adam Welc 
was one of the authors.  Sounds like this may be the same work that 
Stefano is talking about.


cheers

zouqiong wrote:

Hi, Stefano
  I think your idea about to keep a profile for the next run of the
same software is a
good idea. We, people, here always talk about this idea. But in fact, 
few vm

implements
this idea, i am curious about it.


2006/7/28, Stefano Mazzocchi <[EMAIL PROTECTED]>:


zouqiong wrote:
> I am very happy to know a person who has a common goal with me :-)
>
> en, my research focus on how to get more useful information to guide
> compile
> thread
> and garbage collector to optimize the data locality and code quality,
> and my
> interest
> is how to use performance monitor to get profile , and i think few
vitural
> machine use the performance monitor to guide code optimization
> automatic. Of
> course, many paper
> use PMU to guide the analysis.

There is one interesting trick that I learned at an IBM presentation
that was done at MIT last year (apologies, I don't remember the name of
the speaker) about the trick of "persisting" the runtime optimization
information that the JVM discovers after the first few runs of a
particular program.

For server side software, the warm-up time is not a big deal, but for
client side software, it's a huge deal if Eclipse (or your, ehm, firefox
extension written in java ) starts up in 0.5 seconds instead of 5.

The idea, basically, is that the *very first* time an application is
run, the JVM will profile it and optimize the bytecode -> native
compilation and go on doing its normal hot-spot stuff... but then every
now and then, or at shutdown, the JVM will write that info on disk, so
that it will be possible for the *next* run to start with a
"optimization profile" that is, so-to-speak, precompiled and doesn't
have to be inferred from the runtime execution of the program.

There are some issues with the approach, but I think it's a great idea
and nothing that we, as a community, can't find an answer for.

BTW, would be great to have those people here talking to us about those
new JVM tricks they are doing in IBM for J9.

--
Stefano.


-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]








-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm/mmtk] jitrino.jet write barrier design questions

[Robin Garner]

Weldon Washburn wrote:

Alex,
Sorry for taking so long to answer your questions below.  Please see
the response inline.
   Weldon

On 6/27/06, Alex Astapchuk <[EMAIL PROTECTED]> wrote:

AFAIR from the recent thread, to implement WB for MMTk support, I have
to emit calls of

   org.mmtk.plan.PlanLocal.writeBarrier(
   ObjectReference src,
   Address slot, ObjectReference tgt,
   Offset metaDataA, int metaDataB, int mode)
on every PUTSTATIC/PUTFIELD/AASTORE.

I can guess what 'src' is - this is the object being written, right ?
But could you please point me what all other args are ?


A quick description of what all the writeBarrier() fields are:

src
Reference pointer to the object that's getting modified via
putfied/putstatic/aastore

slot
The machine address into which the new reference will be stored

tgt
The reference pointer that will get written into the slot

metaDataA
The difference between the machine address of the "slot" and "src"

metaDataB
I think its intended to be used as some sort of an index into VM
internal (class loader) array struct that holds info on which fields
of a given object are ref ptrs.
MetadataA and B were introduced to solve a problem in the JikesRVM opt 
compiler.  While I'm not sure of the details, they allow the compiler to 
preserve type information across the write barrier call.  Bottom line: 
they're there for the compiler: don't need 'em, pass zeros.


mode
Mode is either PUTASTATIC, PUTFIELD, AASTORE


Can't we go without all the stuff and have only 2 args - an
object being written and the destination class/array/instance ? :-)


These parameters support different styles of barrier.  Some barriers 
remember objects which have been mutated, then scan the object during a 
minor GC; others remember the pointer itself, and avoid having to scan 
(potentially large) objects - hence the need for the src and slot 
parameters.


The mode parameter could potentially allow the two strategies to 
coexist, by saving slots for address arrays (expensive to scan if big) 
and objects for scalars.  I don't think we do it, but it would be a 
shame to make it impossible.



If we did this, ultimately we would end up making some ugly hacks on
MMTK's writeBarrier().  I'd like to avoid this approach until a real
compelling reason surfaces.

For initial bring up, the only write barrier GC I worry about is
mmtk.plan.generational.  The generational GC only uses "slot" for the
actual write barrier.  Since it is a substituting write barrier,
writeBarrier() calls the code that is responsible for actually
scribbling the ref ptr on the Java heap.  The code that scribbles on
the heap is intended to be developed during MMTK port.  Most likely I
will write this code.  The only parameters needed (I think) is "slot"
and "tgt".

I expect this is probably the approach you're taking, but I would do the 
port by targeting the NoGC, MarkSweep, SemiSpace, CopyMS and GenMS 
collectors in that order - the write barrier isn't needed then until the 
very end.



In other words, you can fill in a zero for src, metaDataA, metaDataB
and mode for initial bring up.  But ultimately they have to have
legitimate values to satisfy MMTK writeBarrier() interface.

We also assume that the compiler will probably optimize all the 
parameters away or into registers, so the # parameters shouldn't be a 
performance issue.


cheers
-- Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] GC/VM interface discussion

[Robin Garner]

Weldon Washburn wrote:

On 7/11/06, Robin Garner <[EMAIL PROTECTED]> wrote:

Salikh Zakirov wrote:
> Ivan Volosyuk wrote:
>
>> My idea here, is if the function is required by VM, it can be 
emulated

>> on VM side via "alloc + pin". Of cause, it is not the same as
>> gc_pinned_alloc. The behaviour of GC will be suboptimal in that case.
>>
>
> In fact, gc_alloc_pinned() is not intended to be equivalent to 
gc_alloc()

> followed by gc_pin() for the following reasons
>
> 1) gc_pin() is optional operation, and may be absent in simplest 
collectors
> 2) gc_pin() is not required to succeed, and the GC may decide to 
reject
>object pinning (e.g. based on some memory footprint/performance 
compromise).
>The success of operation must be checked by using 
gc_is_object_pinned()
> 3) gc_alloc_pinned() may be implemented in a much more efficient 
way than
>pinning of arbitrary object, e.g. by introducing the separate 
fixed space
>with size-segregated lists or other malloc-like management 
algorithms.

>
>
Perhaps a more relevant discussion is exactly what
hints/suggestions/requirements the VM and MM give each other on an
allocation request.  MMTk allows the VM to explicitly request allocation
in a specific space, and this can/has been used to implement
optimizations such as pre-tenuring.


Yes!  I like it :)


IMO a good high-performance
interface should at least allow the VM to provide a variety of hints to
the MM.


Yes!  I hope you can help us define these performance hint APIs.


The ones I *know* are useful are:

- Immobile (object will never move)
- Mature (for pretenuring, full-heap GC ignores)


I could speculate about a bunch of other useful hints (eg hot/cold, 
thread local, large/small ...), but probably nowhere near exhaustively - 
probably best to simply create the mechanism, define a couple of hints 
and leave it open for future use.


-- Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] GC/VM interface discussion

[Robin Garner]

Xiao-Feng Li wrote:

On 7/11/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:
> > > > GCExport void gc_write_barrier(Managed_Object_Handle 
p_base_of_obj_with_slot);

> > > > This is a difficult API.  For starts, it only applies to write
> > > > barriers written in C/asm.  Its not usable by MMTk.  "C" Write
> > > > barriers that also need the ref ptr that gets written to heap 
will
> > > > need to call a different API.  That is, unless the JIT can 
*inline*
> > > > random chunks of C/asm and optimize across the call 
boundary.  A hard
> > > > and ugly thing to have to do!  Since old ORP generational GC 
and also
> > > > SableVM GC only use the above API, my vote is to leave this 
API "as

> > > > is" for now.
> > >
> > > Agreed. This is too specific to be a reasonable API.
> >
> > I would like to keep this API. As we talking about C interfaces, 
this

> > C interfaces can be used by C garbage collector. I know two places
> > where the function feets perfectly: object_clone and array_copy. 
If  C

> > garbage collector provides that function the code is simple - do the
> > copying and call the function. If not, number of per-slot replacing
> > write barriers should be called, which is quite slow for C garbage
> > collector.
> > In all other places in VM use of this function should be 
discouraged.

>
> I would not suggest to define specific API for specific optimization
> situations. This API usage model can be implemented in other API  by
> overloading a parameter or an extra parameter if it is really
> desirable.

The is not suitable from performance point of view. Extra parameter if
checked each time will degrade performance substantially. Write
barrier is performance critical peace of code and should be very
simple and optimal. This API can also be declared as _optional_. If
the
GC doesn't export this optimized function, VM can fallback to general
case when all slots
of the object should be updated by separate write barrier calls.



My experience was that write barrier performance is decided by the
condition checking(s) in fast path. Have you any data showing this API
is important for certain benchmarks?

Thanks,
xiaofeng

There are two papers I can think of off the top of my head that are 
relevant to this question. 


_Blackburn, S. M._ and Hosking, A. L. *Barriers: Friend or Foe?* /
The 2004 International Symposium on Memory Management (ISMM 2004) 



/_Blackburn, S. M._ and McKinley, K. S.  *In or Out? Putting Write 
Barriers in Their Place* /
The 2002 International Symposium on Memory Management (ISMM 2002) 


/
Both available from 
http://cs.anu.edu.au/~Steve.Blackburn/pubs/abstracts.html



For best performance, the fast path of both the allocation sequence and 
the write barrier should be inlined into the mutator, and especially for 
the write barrier the slow-path needs to be out-of-line (see "In or Out" 
paper for details).  I gather the usual approach is to provide 
hand-crafted IR for each operation, so perhaps there should be an 
additional API function along the lines of


void *get_IR()

where the compiler can request a chunk of IR to optimize certain 
operations - and if the MM can't provide it, simply compile a function call.



Also for completeness, read barriers should be considered.  While MMTk 
CVS head doesn't currently use this, various people have patches that 
implement them, and we are keen to add this to MMTk real soon now.



Lastly, and this may or may not be appropriate to consider in the 
context of DRLVM, consideration should be given to barriers on statics 
and non-pointer fields.  The various transactional approaches to 
synchronization make use of these, and IMO this is going to be a big 
area in the not too distant future.   OTOH, this is probably such a 
pervasive change that it would be best to engineer a new VM with these 
kinds of barriers in place from the start.


-- Robin
/
/

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] GC/VM interface discussion

[Robin Garner]

Salikh Zakirov wrote:

Ivan Volosyuk wrote:
  

My idea here, is if the function is required by VM, it can be emulated
on VM side via "alloc + pin". Of cause, it is not the same as
gc_pinned_alloc. The behaviour of GC will be suboptimal in that case.



In fact, gc_alloc_pinned() is not intended to be equivalent to gc_alloc()
followed by gc_pin() for the following reasons

1) gc_pin() is optional operation, and may be absent in simplest collectors
2) gc_pin() is not required to succeed, and the GC may decide to reject
   object pinning (e.g. based on some memory footprint/performance compromise).
   The success of operation must be checked by using gc_is_object_pinned()
3) gc_alloc_pinned() may be implemented in a much more efficient way than
   pinning of arbitrary object, e.g. by introducing the separate fixed space
   with size-segregated lists or other malloc-like management algorithms.

  
Perhaps a more relevant discussion is exactly what 
hints/suggestions/requirements the VM and MM give each other on an 
allocation request.  MMTk allows the VM to explicitly request allocation 
in a specific space, and this can/has been used to implement 
optimizations such as pre-tenuring.  IMO a good high-performance 
interface should at least allow the VM to provide a variety of hints to 
the MM.


-- Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm/mmtk] jitrino.jet write barrier initial implementaion

[Robin Garner]

Weldon Washburn wrote:

On 7/10/06, Alex Astapchuk <[EMAIL PROTECTED]> wrote:

Weldon,

I just commited http://issues.apache.org/jira/browse/HARMONY-816
into JIRA.


Thanks.  I will look at it today.



It contains the changes for Jitrino.JET:

- magics support for MMTk
- write barriers for Java for MMTk
- a simple test to check the things are alive

I also have few questions on unboxed package and some MMTk
internals, that I was unable to resolve myself (I used javadocs
from http://jikesrvm.sourceforge.net/api/) and I would
appreciate a help on resolving them.


The main obstacle is that writeBarrier method in PlanLocal is
instance method.
I failed to find a way how to get the proper instance during
runtime - and this is the first question - how to get the
proper instance of PlanLocal ?


Yes.  We will need to add APIs for the jit to get the ref ptrs to a
few MMTk java objects.  My guess is that MMTk can allocate these
objects in "immortal space" which is never moved, never collected thus
the ref pointer is always valid.

The latest CVS head for MMTk clarifies/complicates this even more:)

Steve has refactored the PlanLocal into a 'MutatorContext' and a 
'CollectorContext' object.  The Mutator Context is a per-thread data 
structure, and the collector context is per-GC-thread (MMTk collectors 
are parallel).


Currently, I've made a presumption that there must be some
static method to get it.


We need to discuss the approaches.  The API could be a static java
method but this requires the JIT to call out to java code while
jitting which might not be a good idea.  Another approach is to use
the existing class loader API to return the address of a java static
variable.  Its a proven/tested existing interface.  Thoughts?


What about a pointer off the per-thread data structure ?

I used a simple fake stub to test the implementation
(it's in PlanLocal.java attached to the JIRA).

Also, there are few questions on unboxed package:

  - Address::max() - what is 'maximul allowable Address' ?


I think its 2**32 or 2**32-1 or something like that.



2**

   A highest valid addres in heap ? Or simply ~(void*)0 ?

The above seems reasonable.  The rest of your questions I will think
about and get back to you.

Address::min() should be 0.

  - What's the diff between diff() and sub() ? diff()=='abs(sub())' ?

(sub has actually been replaced by minus in the past few weeks)

Diff() returns an Offset (a signed quantity) that represents the 
distance between two addresses.  sub() (or minus() now) subtracts an 
offset from an address returning an address.  The difference is in the 
types they take and return, and interpretation of arguments as signed or 
unsigned.

  - Word::one() - what's this ? (Object)1 ?

Almost (modulo types).  It's the value 1, as type Word.

  - what is AddressArray::getBacking() ?

No idea.  Current MMTk CVS head doesn't have it.

  - rshl, rshr, rsha - they shift, but what and how
   exactly they're shifting ?
Right-shift arithmetic (ie with sign extension) and right-shift logical 
(no sign extend).  Never heard of rshr.

  - what are exact values for mode argument in writeBarrier ?
   Currently, I'm simply using 0/1/2.

these are the *_BARRIER constants in org.mmtk.utility.Constants.

cheers,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [drlvm/mmtk] jitrino.jet write barrier initial implementaion

[Robin Garner]

Weldon Washburn wrote:

On 7/10/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:

On 7/10/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> On 7/10/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:
> > One question, why do we need write barriers for 'putstatic' 
bytecode?

> > This fields appear in root set and does not represent heap to heap
> > references. There is no need to add write barriers to this entities.
>
> Good question.  The short answer is because MMTk makes zero
> assumptions about the VM automatically enumerating all the static
> locations holding ref ptrs.   DRLVM makes the assumption that statics
> will be part of the root set.  We really don't have hard data at this
> point to show us the best approach yet.   I vote for leaving it in for
> now. This is really the kind of GC/VM integration issue I expect the
> MMTk port force us to deal with.

I think nothing is done without the reason. Looks like the static
fields in Jikes live somewhere in java heap. The code will make sense
if DRLVM will do the same, otherwise the write barriers will possibly
confuse implementation of MMTk, as the changed values are reside
outside the heap.


Its not clear what you are trying to say.  In any case, it should be
straight forward to sort out putstatic ref ptrs once we get to the
debug stage.

Static fields in JikesRVM live in the JTOC and are enumerated as roots, 
just like in drlvm. 

MMTk simply requires all pointers from outside the heap into it to be 
enumerated as roots.  If statics were part of 'class' objects in the 
heap, or as an Object array, then the existing write barrier should 
function correctly.  In fact MMTk has no notion of 'static' fields, as 
it aims to be (target) language independent.


cheers,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] MMTK porting issues, take one

[Robin Garner]

Rodrigo Kumpera wrote:

On 6/2/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:

All,

Perhaps the following is already covered in documentation.  If this is
the case, please tell me where to find it.  Below are some initial
questions regarding porting MMTK to DRLVM.

A question about org.vmmagic.pragma.InlinePragma class.  The comments
in the code says, "This pragma indicates that a particular method
should always be inlined by the optimizing compiler."  Just to be
clear, will there be any correctness issues if a non-optimizing
compiler does not do any inlining?


AFAIK the inline pragmas are used to clearly define the alocation fast
path for the optimizing compiler.
Not just the allocation fast path, but that's the idea.  The pragmas are 
probably most important in the write barriers, where both the 
InlinePragma and NoInlinePragma are important (see "In or Out? Putting 
write barriers in their place.", Blackburn & McKinley, ISMM '02). 



It looks like we have to get Jitrino.JET to generate "funny" binaries
for all the classes in the org.vmmagic.unboxed package.  Is this
correct?  Are there any other packages that bend the type safety rules
we need to worry about?


The magic types are how the MMTK does pointer operations and unsigned
math, the compiler must emit special code for the methods and statics
of these classes. Actually it's pretty easy to emit such code from a
non-optimizing compiler.


It looks like org.vmmagic.unboxed.Extent needs the JIT to specifically
emit instructions that do unsigned arithmetic.  Is this correct?

Correct.


A question on org.vmmagic.unboxed.ObjectReference class --- there is a
comment that says, "Note: this is a JikesRVM specific extension to
vmmagic."  But a grep of MMTK source shows hundreds of uses of this
class.  Does the comment mean that MMTK should not use ObjectReference
class?  Or maybe in the future MMTK will not use ObjectReference
class?

I think the comment is actually incorrect.

cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] MMTk vmmagic classes, can someone w/ compiler experience help me?

[Robin Garner]

Robin Garner wrote:

Weldon Washburn wrote:

All,

I am hoping someone who has worked on compilers can actually do the
JIT modifications.  I don't have much experience in compilers.

I am trying to get MMTk write barriers integrated into Harmony DRLVM.
I came up with the following scheme.  I don't know if it is correct.
It would be great if someone from the MMTk crowd looked at it.  If it
helps, I can also post this message on Jikes/MMTk mailing list.

Build a shim between the DRLVM class loader and Jitrino.JET.  The shim
would contain a lookup table that would map _local_ variables of
specific types to int.  In particular, the shim would re-map local
variables of the below types to int:

Address
Extent
Offset
Word

The reason for the shim is to avoid modifying the class loader.  This
should reduce the maintenance burden.

Java source code that creates objects of the above classes is a now a
problem.  For example, Java source code that does:

int xx = 33;
Address a1 = new Address(xx);

MMTk never creates instances of the unboxed magic types, so this 
shouldn't be a problem.


Oh, and 'int' is a bad choice, since it is always 32 bits.  These types 
should be whatever the natural word length of the target architecture is.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] MMTk write barrier -- questions about which writeBarrier() API to use

[Robin Garner]

Weldon Washburn wrote:

All,

Perhas the MMTk crowd knows the answer to the following questions.

Can I simply not use
org.mmtk.plan.PlanLocal.writeBarrier(ObjectReference src, Address
slot, ObjectReference tgt, Offset metaDataA, int metaDataB, int
mode);?

Instead, I want to only use writeBarrier(ObjectReference src, Offset
srcOffset, ObjectReference dst, Offset dstOffset, int bytes);.  Will
this be a problem?
The two writebarriers are for separate cases.  The first is a putfield 
write barrier, the second is a special case for a reference array copy.


Questions about the incoming args:

ObjectReference src
From the JITs perspective, an ObjectReference is indistinguishable
from a java.lang.Object.  Is this true?  False?

Yes, but MMTk assumes that its target language is not necessarily Java.  
An ObjectReference is a pointer to a heap object, whatever that may be 
in the language being managed.

Address slot
When is  "Address slot" argument actually created?  Does this Address
object live long enough such that its "value" field needs to be
updated following a copying GC?  Is the answer the same for both Jikes
and the Rotor ports?
A write barrier should never be invoked on an object that is being 
copied.  An Address is an unboxed type, so objects of that type are 
never created.


Offset srcOffset

In DRLVM, the classloader resolves a field offset once and it never
changes.  Does it make sense for the classloader to create all the
Offset objects during load time?  Initially, I want to create these
objects _outside_ the formal java heap to have tight control over
object movement and deletion.   Basically, I don't want the Offset
object to ever move or ever be deleted during the initial stages of
MMTk integration.
Offset objects are never created.  Think of an offset as a primitive 
type with methods.


A question about how jikesrvm-2.4.4/MMTk handles objects that are not
inside the offical heap.  Are these objects simply ignored?  I know
that ECMA CLI spec requires that objects which are not in the official
heap must be ignored.  I simply don't know if this requirement is
incorporated in 2.4.4/MMTk source base.
Any object that MMTk encounters must be in the heap that it manages.  In 
JikesRVM/MMTk, there are a minimum of 4 regions of the heap, VM, 
Immortal, LOS and then any plan-specific region.  I think the objects 
you are referring to would be in the VM space ??? 


While it looks like a lot of work to get DRLVM to generate Offset
object properly, it looks like even a bigger job to modify MMTk to
replace Offset class with an "int" that holds a given field's offset.
Any opinions on this statement?


Not true, IMO.  The JikesRVM compiler replaces Offset "objects" with a 
primitive type of the natural word length of the machine.


Hope this helps,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] MMTk vmmagic classes, can someone w/ compiler experience help me?

[Robin Garner]

Weldon Washburn wrote:

All,

I am hoping someone who has worked on compilers can actually do the
JIT modifications.  I don't have much experience in compilers.

I am trying to get MMTk write barriers integrated into Harmony DRLVM.
I came up with the following scheme.  I don't know if it is correct.
It would be great if someone from the MMTk crowd looked at it.  If it
helps, I can also post this message on Jikes/MMTk mailing list.

Build a shim between the DRLVM class loader and Jitrino.JET.  The shim
would contain a lookup table that would map _local_ variables of
specific types to int.  In particular, the shim would re-map local
variables of the below types to int:

Address
Extent
Offset
Word

The reason for the shim is to avoid modifying the class loader.  This
should reduce the maintenance burden.

Java source code that creates objects of the above classes is a now a
problem.  For example, Java source code that does:

int xx = 33;
Address a1 = new Address(xx);

MMTk never creates instances of the unboxed magic types, so this 
shouldn't be a problem. 


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Fwd: [DRLVM] proposal to port MMTK to drlvm

[Robin Garner]

Weldon Washburn wrote:

oops, I forgot to cc:

-- Forwarded message --
From: Weldon Washburn <[EMAIL PROTECTED]>
Date: May 24, 2006 11:23 AM
Subject: Re: [DRLVM] proposal to port MMTK to drlvm
To: [EMAIL PROTECTED]


On 5/24/06, Daniel Feinberg <[EMAIL PROTECTED]> wrote:

that is cool, so the other thing i was thinking about is that MMTK is
written for JikesRVM which is a meta-circular java implementation. AKA
it is written in java with some magic for the low level mem stuff. I
am not sure how that would integrate into Harmoney and the DRLVM. I
dont think it would be hard to port it to another language or we could
write some sort of interface to bridge the two. Not sure what are your
ideas?


hmm somewhere I heard that MMTK had been ported to Microsoft
Rotor.  I know Rotor is a vm written in C/C++.  If this port is under
an Apache agreeable license, then we could look at this code.  If no
MMTK-to-C interface that is compatible with Apache license exists
then, of course, we will need to build one.  My preference would be to
keep it the interface combinations to C and Java.  I would rather not
bog down MMTK/HarmonyDRLVM with any C++ distractions initially.  I
think the initial focus is functionality, cleaning up the C interface,
then performance.
The port to rotor was done by Andrew Gray at ANU, and was based on my 
work integrating MMTk into C-based runtimes.  The approach used was to 
apply a source code transformation to turn the MMTk 'magic' into native 
methods (using CNI) on primitive types, and compile MMTk with gcj.  It 
is described in detail in my honours thesis.


The source transformer code is licensed as public domain, but would need 
considerable modification to work on the current MMTk code base.  A far 
better approach would be to implement the vmmagic types in gcj.  The 
bulk of the C code in the interface to Rotor is specific to rotor, and 
would need to be re-implemented for DRLVM anyway.


The MMTk <-> VM interface is much cleaner these days: as Daniel points 
out, MM_Interface defines the JikesRVM -> MMTk interface, and MMTk's 
interface to the vm is the package org.mmtk.vm.  MMTk is compiled to a 
JAR file against stubs for this package, and the directory ext/JikesRVM 
contains the JikesRVM-specific implementation of the interface.


cheers

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [DRLVM] proposal to port MMTK to drlvm

[Robin Garner]

Weldon Washburn wrote:
Good point.  Which  MMTK code base should we work with?  Where is it 
located?


I know one version of MMTK is bundled with the Jikes source code.   I
do not know if it is the best version to start this project.   There
is a Jikes download link on the page:
http://jikesrvm.sourceforge.net/download/index.shtml.  The version on
this page is jikesrvm-2.4.4.tar.gz.  Does anybody know of a better
version to start with?



On 5/25/06, bootjvm <[EMAIL PROTECTED]> wrote:


Weldon,

Where can I find the MMTK source?  I've not been
able to locate it where I thought it was supposed to
be on SourceForge.

Thanks,


Dan Lydick


> [Original Message]
> From: Weldon Washburn <[EMAIL PROTECTED]>
> To: 
> Date: 5/24/06 1:09:03 AM
> Subject: [DRLVM] proposal to port MMTK to drlvm
>
> Folks,
>
> There were several interesting email chains about Harmony VM and MMTK
> last year.  This topic died in large part because there was no JVM.
> Since then, several JVMs have been donated.  I volunteer to do an
> initial investigation of porting MMTK to the recent DRLVM donation.
> From a quick grep of the code, it appears that write barriers are only
> partially implemented.  We will need to make write barriers functional
> before many of the features of MMTK can be used.   Thoughts?
>
> Weldon
>
>
>
> --
> Weldon Washburn
> Intel Middleware Products Division
>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>







MMTk resides in the jikesrvm code base for the moment, 
jikesrvm.sourceforge.net. 

CVS head is the best version to work with.  I'm happy to answer any 
questions you might have about interfacing to it.


cheers
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Split the list ?

[Robin Garner]

Has anyone else considered the possibility of splitting the harmony-dev 
list ?  As a memory manager/VM hacker, I'm interested to keep up with 
the high level happenings in the harmony world, but not necessarily with 
the low-level details of bugs in the class library (and so forth). 

What do people thing about creating, say, a harmony-classlib list, and 
leaving the main harmony-dev list for less in-depth discussions ?


cheers,
Robin

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Summer Of Code 2006 - Lets get Harmony involved

[Robin Garner]

Tim Ellison wrote:


Sanket Sharma wrote:
 


Performance analysis sounds interesting...

Any links/pointers to get started?
   



I suggest that you look around for some simple well-written performance
benchmarks for Java and see if you can get them running on Harmony.

 

Not sure whether they qualify for _all_ of Tim's criteria (:P), but it 
would be great to see if anyone could get the dacapo benchmarks (or a 
subset of them) working with the harmony class libraries.  I'm actively 
updating them at the moment and trying to get the last couple working 
with the classpath libraries, so the timing's good.


Expect a new (beta) release within the week.

cheers
Robin

http://www-ali.cs.umass.edu/DaCapo/index.html (click on Dacapo Benchmarks)

-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [classlib] performance tips

[Robin Garner]

Mikhail Loenko wrote:


Hello

May be it is obvious and everybody knows it from babyhood, but anyway...

Everybody knows that this two examples of code do the same:
class test {
   public Object field = null;
}

and

class test {
   public Object field;
}

But if you disassemble these two classes, you'll see that the first example
has a 6 instruction constructor:

  0: aload_0
  1: invokespecial #1; //Method java/lang/Object."":()V
  4: aload_0
  5: aconst_null
  6: putfield #2; //Field field:Ljava/lang/Object;
  9: return

while the second one has only 3 of them:
  0: aload_0
  1: invokespecial #1; //Method java/lang/Object."":()V
  4: return

So having explicit assignments of default values slows down constructor.

Thanks,
Mikhail
 


Mikhail,

Is this something a bytecode compiler should be able to optimize away, 
or is there something in the JLS that requires the additional bytecodes 
to be generated ?  Have you measured the impact (I suspect it is 
negligible).


cheers

Re: Building choices (was: Re: Code contribution to harmony)

[Robin Garner]

Matt Benson wrote:


--- Ashish Ranjan <[EMAIL PROTECTED]> wrote:

 


that is the most convincing argument till now.   :-)
   



+1 from an Ant PMC member.  That logic is irrefutable.
:)

-Matt


 

What about cross-compilation/cross-building ?  If harmony is to be 
successful in its goal of wide portability I would expect that the build 
process would make cross-builds almost as easy as native builds.


-- Robin


bye :-)
Ashish Ranjan
India
[EMAIL PROTECTED]

On 11/23/05, Graeme Johnson
<[EMAIL PROTECTED]> wrote:
   


Tim Ellison <[EMAIL PROTECTED]> wrote on
 


11/21/2005 07:17:16 AM:
   


Andrey Chernyshev wrote:
   


On 11/15/05, Tim Ellison
 


<[EMAIL PROTECTED]> wrote:
   


In the end we decided to go with a
   


'conventional' native code tool set
   


for the native source, and 'conventional' Java
   


code tools for the Java
   


source.  People just felt more comfortable
   


with that.
   


Do you think we are missing out on something
   


;-) ?
   


Well, I can see a few potential issues with
 


such "mixed" approach:
   


- In order to contribute, people would have to
 


learn both building
   


technologies - Ant and make, someone may give
 


up.
   


I don't see a great advantage to asking people
   


to learn 'cpptask' rather
   


than 'make'.  I would suggest that many more C
   


programmers are familiar
   


with 'make' already, so we are not asking them
   


to learn something new.
   


[snip]
   

'make' also simplifies the bootstrapping issue. 
 


When you are doing the
   


initial port of the VM to a new platform, and you
 


don't have java
   


running yet, having your build instructions
 


encoded in Ant is problematic.
   


Relying on the availability of a previous java
 


port to get the Harmony
   


VM building seems like a questionable porting
 


story.  'make' of one flavor
   


or another is pretty much universally available,
 


and seems like the
   


pragmatic choice for building C code.

Graeme Johnson
J9 VM Team, IBM Canada.

 






__ 
Yahoo! FareChase: Search multiple travel sites in one click.

http://farechase.yahoo.com
 

Re: half-baked idea? j2me

[Robin Garner]

>
> On Nov 4, 2005, at 3:20 AM, Robin Garner wrote:
>
>> Geir Magnusson Jr. wrote:
>>
>>
>>>
>>> On Nov 1, 2005, at 9:05 PM, Robin Garner wrote:
>>>
>>>
>>>> Actually my colleagues at ANU and I were remarking last week
>>>> that  all the recent discussion on the Harmony list (configure
>>>> scripts,  packed structs etc etc) were close to being proof that
>>>> Java was the  easier way to go.
>>>>
>>>
>>>
>>> C'mon... you still have to deal with that with a Java based VM
>>> because you still need the bootstrap stuff...
>>>
>>> geir
>>>
>>>
>>>
>> Yes, but only in a tiny percentage of the code, only for a handful
>> of API calls, and none of it performance critical.
>>
>
> And all of it requiring configure scripts?  :)
>
> geir

Actually probably little enough problematic code that you could write a
separate version for every platform/compiler for comparable effort to
writing the configure script.

cheers

Re: half-baked idea? j2me

[Robin Garner]

Geir Magnusson Jr. wrote:



On Nov 1, 2005, at 9:05 PM, Robin Garner wrote:

Actually my colleagues at ANU and I were remarking last week that  
all the recent discussion on the Harmony list (configure scripts,  
packed structs etc etc) were close to being proof that Java was the  
easier way to go.



C'mon... you still have to deal with that with a Java based VM
because you still need the bootstrap stuff...

geir


Yes, but only in a tiny percentage of the code, only for a handful of 
API calls, and none of it performance critical.

Re: half-baked idea? j2me

[Robin Garner]

Archie Cobbs wrote:


Robin Garner wrote:

Actually my colleagues at ANU and I were remarking last week that all 
the recent discussion on the Harmony list (configure scripts, packed 
structs etc etc) were close to being proof that Java was the easier 
way to go.



Here's some idle speculating about writing a JVM in Java...

Start by asking this question: if you could design a new language
expressly for the purpose of implementing JVM's, what would that
language look like?

Java is almost the right language.. but not quite. You need to
be able to do C-like stuff as well.

One can imagine something that is mostly like Java, but has some
additional features that allows C like functionality, for example:

- Augment the Java type system with C-like "structs". These are
  like Java objects in that they can be allocated on the Java heap
  (as an option) but have no Object header (you can't synchronize
  on them directly and they have no associated Class). Then the
  in-memory representation of an Object is a special case of one
  of these structures, containing a lockword and vtable pointer.

- Define a new "word" primitive type that corresponds to the
  machine-specific word size (i.e., 32 or 64 bit unsigned int).
  Corresponds to SableVM's _svm_word and JC's _jc_word.

- Language would include primitives for compare-and-swap of a word,
  memory barriers, etc.

- The language would include the ability to cast between any types
  as you can do in C (e.g., struct -> Object, word -> Object pointer).

- Allow C function calls to be expressed in the language, passing
  as parameters any Java type, or a struct. This "compiles" directly
  into a C function call using the platform's normal C calling
  conventions.

- Extend the class file format in a corresponding manner.

Call this language "Java++" (or whatever). Then the 95% of the JVM
can be written in this language.. and 95% of that would be normal Java.

This is exactly how we see the dialect of Java that MMTk is written in.  
The non-java extensions are the org.vmmagic classes.  The key difference 
is that our types are represented as 'unboxed' objects, which gives us 
more flexibility to define operations on them.


cheers

Re: half-baked idea? j2me

[Robin Garner]

Rodrigo Kumpera wrote:


On 11/1/05, Robin Garner <[EMAIL PROTECTED]> wrote:
 


On 11/1/05, Robin Garner <[EMAIL PROTECTED]> wrote:
 


Rodrigo Kumpera wrote:

   


AFAIK IKVM, sablevm and jamvm all run on portable devices.

Developing a j2me jvm is not as easier as it seens, first, the
footprint and execution performance must be really optimized, so
expect a LOT of assembly coding.


 


Back to the language wars again :)  This does not necessarily follow.
Try googling for the 'squawk' VM - they had a poster at OOPSLA last
week.  This is a java-in-java virtual machine targetted at embedded
devices.  The core VM runs in 80KB of memory.  Device drivers are all
written in Java.

   


Robin,

With a java-in-java VM even if you don't write directly in assembly
you still need to generate machine code with java anyway, and that
will look a lot like asm (JikesRVM baseline JITer for example). With
C, for example, you can get away using just an interpreter.
 


My mistake, obviously.  When you said "performance must be really
optimized, so expect a LOT of assembly coding", I assumed you were saying
that large chunks of the VM would need to be written in assembler in order
to get adequate performance.

So what _was_ the point you were making ?

cheers


   



I was just trying to say that a decent j2me VM is not as simple as
David suggested. Not that C or Java would be more suited to implement
it. As a matter of fact, I think that java-in-java VMs can be as good
as C/C++ based JVMs or better.

But one thing is hard to deny, a simple JVM, like bootJVM, is a lot
easier to write in C than in java (not using an AOT compiler). And
that was my point, C/C++ sounds to be the easy path to start with.
 

Actually my colleagues at ANU and I were remarking last week that all 
the recent discussion on the Harmony list (configure scripts, packed 
structs etc etc) were close to being proof that Java was the easier way 
to go.


Another data point (FWIW) - joeq, excluding the compiler and the class 
library interface comes in at ~39,000 lines of code.  bootJVM is already 
over 50,000.  I know that KLOC is a pretty bogus measure of complexity, 
but it certainly says _something_.  And Joeq is a fully functioning VM.


cheers