date:20061030

Yeah! No problem, I was just thinking a zip would not look as
transparent and safe. I'd turn the images from the other doc issue into
an archive as well.

Thank you, 
Nadya Morozova

-Original Message-
From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, October 31, 2006 5:20 AM
To: harmony-dev@incubator.apache.org
Subject: Re: [doc][drlvm] new docs added - JIT compiler

is there any chance you could put a zip of the images into the jira, 
rather than a  dozen or so images?  That's actually what turned me off 
from one of the other doc issues - I said to myself "oh, heck, I don't 
want to download each of those separate thingies... I just want a zip or

tgz of them... easier to handle..."

So please? :)

geir

Morozova, Nadezhda wrote:
> All, 
> 
> New documents have been added to HARMONY-2003. These are a description
> of the Jitrino.OPT and .JET compilers, with a supplementary doc on the
> pipeline management framework inside the JITs. The docs describe the
> architecture, processes and usage of the components.
> 
>  
> 
> It would be great if somebody took a look, expressed an opinion, and,
if
> favorable, committed to the website.
> 
> Your review or any other feedback are most welcome,
> 
>  
> 
> Thanks, 
> 
> Nadya Morozova
> 
>  
> 
> PS: this JIRA is not the only one unresolved doc issue. Find more
useful
> pending patches in our database :-)
> 
>

Re: [general][snapshot] Windows installer

2006-10-30 Thread Alexey Petrenko


Installer is great!
Thanks, Tim.

SY, Alexey

2006/10/30, Tim Ellison <[EMAIL PROTECTED]>:

I thought it would be fun to see what a Windows installer would look
like for Harmony.

So this morning, just for kicks, I was playing with NSIS
(http://nsis.sourceforge.net/) and produced a prototype installer very
easily (kudos to them).  I'll put the source for it here [1], and you
can download an actual Harmony installer from here [2].

Caveat Emptor:  Did I mention it is a prototype already?  Feel free to
play with it.

It's easy to change the sets of files to install, right for now I
created base runtime, Class library sources, Java development, and HDK.

[1]
http://svn.apache.org/viewvc/incubator/harmony/enhanced/tools/trunk/windows_installer/
[2]
http://people.apache.org/~tellison/Apache%20Harmony%20Installer%20(r468731).exe


Regards,
Tim

--

Tim Ellison ([EMAIL PROTECTED])





--
Alexey A. Petrenko
Intel Middleware Products Division

Re: [drlvm] Class unloading support

2006-10-30 Thread Robin Garner

Weldon Washburn wrote:

On 10/27/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:

Weldon Washburn wrote:
> Steve Blackburn was in Portland Oregon today.  I mentioned the idea of
> adding a  reference pointer from object to its j.l.Class instance.  
MMTk

> was
> not designed with this idea in mind.  It looks like you will need to 
fix
> this part of MMTk and maintain it yourself.  Steve did not seem 
thrilled

at
> adding this support to MMTk code base.

Actually I think the answer may have been a little garbled along the way 
here: MMTk is not a memory manager *for* Java, it is simply a memory 
manager.  We have been careful to eliminate language-specific features 
in the heap that it manages.  MMTk has been used to manage C# (in the 
Rotor VM) and was being incorporated into a Haskell runtime until I ran 
out of time.

Therefore, MMTk knows nothing about the concept of class unloading, or 
java.lang.Class.

How does MMTk support class unloading then?

MMTk has no special support for class unloading.  This may have 
something to

do with the entire system being written in Java thus class unloading come
along for free.  If there needs to be a modification to support special 
case

objects in DRLVM, someone will need to fixup MMTk and provide onging
support of this patch in Harmony.  I have zero idea how big this effort
would be.   It would also be good to hear what the impact will be on GCV5.

MMTk implements several algorithms for retaining the reachable objects 
in a graph and recycling space used by unreachable ones.  It relies on 
the host VM to provide a set of roots.  It supports several different 
semantics of 'weak' references, including but not confined to those 
required by Java.

If you can implement class unloading using those (which the current 
proposal does), then MMTk can help.

If you want to put a pointer to the j.l.Class in the object header, MMTk 
will not care, as it has no way of knowing.  If you put an additional 
pointer into the body of every object, then MMTk will see it as just 
another object to scan.

Remember MMTk is a memory manager, not a Java VM!

Conversely, supporting some exotic class unloading mechanism in MMTk 
shouldn't be hard and wouldn't deter me from trying it out.  If (as a 
wild idea) you wanted to periodically scan the heap, and count all 
references to each classloader, you could implement this with very 
little work as a TraceLocal object, and then extend the GC plan you 
wanted with an additional GC phase that would periodically do one of 
these scans after a major GC (for example).

cheers

Re: [drlvm][em] Value-profiling implementation is avaible.

2006-10-30 Thread Pavel Ozhdikhin


Thanks, Yuri! I'll try your patch and get back to you here in the list.

-Pavel


On 10/31/06, Yuri Kashnikoff <[EMAIL PROTECTED]> wrote:


> I'm currently condidering different ways of devirtualization improvement
in
> the JIT compiler. There are many of them and value profiling is one of
the
> most promising. I would be happy if we could work together on making
value
> profiling useful in Jitrino.OPT. Currently devirtualizer in JIT can use
edge
> profile and method hotness profile (in addition to class hierarchy
> analysis and preexistence) to guess about receiver type but value
> profile must be more precise.
It would be wonderful.

> I'd like to look into your patch and play a little with it - could you
> please put your code into JIRA and announce its number here?
http://issues.apache.org/jira/browse/HARMONY-2012
In this pathch only the EM implemenation and the realisation of value
profiler.
Thanks to Mikhail Fursov for his Edge profiler and for His Great help
to me. Thank You, Mikhail. =)

RE: [build] Building on Eclipse - FYI

2006-10-30 Thread Konovalova, Svetlana

Nadya,

AFAIU the given page is purely classlib-oriented...though I might be
wrong here. 
It provides info on how to set up Eclipse to develop Java code in
Harmony and IMHO doesn't include any tips applying to harmony code in
general.
I absolutely agree with you that we should have one page describing how
to work with harmony code in Eclipse. It goes without saying that we
need to update info and the help from engineers' side is of great
importance here. You might know that there is a brief webcast for those
who want to see a step-by-step guide to configuring Eclipse. On the one
hand, it's great, but on the other hand, many screenshots can quickly
become outdated. IMHO it's rather complicated to update the video. What
would you say? Should we keep it as it is?

Cheers,
Sveta

-Original Message-
From: Morozova, Nadezhda [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 6:45 PM
To: harmony-dev@incubator.apache.org
Subject: RE: [build] Building on Eclipse - FYI

Sveta,
I've taken a brief look at the patch, and I like most of your
corrections. The page looks neater this way, and holds important data.

One question: can't we say that some of the tips given on the page apply
to harmony code in general, not just classlib? So a side idea would be
to have one page: how to work with harmony code in Eclipse. What do you
say? 

Thank you, 
Nadya Morozova

-Original Message-
From: Konovalova, Svetlana [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 6:23 PM
To: harmony-dev@incubator.apache.org
Subject: RE: [build] Building on Eclipse - FYI

Sian,

Taking into consideration your comments, I've opened a new JIRA issue
[http://issues.apache.org/jira/browse/HARMONY-2009] and have created a
patch for the page "Developing Apache Harmony Class-library Code with
Eclipse". Would be great, if you find a chance to look it through. 
Hope we'll continue working at developing this aspect of documentation.
:)

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED] 
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian

On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> >

Re: [drlvm][em] Value-profiling implementation is avaible.

Hi Yuri,
Since you invited questions, here are a few from me...

What is the basic profiling mechanism...instrumentation or sampling?
What are the different types of profiles we are capturing ?
How are the profiles currently being persisted?

Is there any existing documentation on this?

Have you given some thought to the training set of applications you will
use?

Thanks,
Rana

On 10/30/06, Yuri Kashnikoff <[EMAIL PROTECTED]> wrote:

2006/10/30, Mikhail Fursov <[EMAIL PROTECTED]>:
> Hi Yuri.
> Value profile is a profile that is really needed for set of JIT
> optimizations.
> What do you want to improve in JIT exactly?
First of all I want to imporove the devirtualization, using value
profiling feedback. And now I found that some other optimization could
use value profling, if it is interesting I can put some ideas here for
considering.

For Geir and other non-Jit-nerds (I'am not JIT-nerd alredy but I'am
trying to become one of them), I'll try to explain what the value
profiling is and how it could be used for increasing JIT performance.

Devirtualization of virtual calls.
We have some virtual calls. And we know that in some program
realisation of some method A used more often (for examle 1000 times)
and antoher realisation of this method B used only 10-20 times. So
this information could be usefull for recompiling the Hot Method A.
And Value Profiler brings this information to JIT.

Re: [drlvm][em] Value-profiling implementation is avaible.


I'm currently condidering different ways of devirtualization improvement in
the JIT compiler. There are many of them and value profiling is one of the
most promising. I would be happy if we could work together on making value
profiling useful in Jitrino.OPT. Currently devirtualizer in JIT can use edge
profile and method hotness profile (in addition to class hierarchy
analysis and preexistence) to guess about receiver type but value
profile must be more precise.

It would be wonderful.


I'd like to look into your patch and play a little with it - could you
please put your code into JIRA and announce its number here?

http://issues.apache.org/jira/browse/HARMONY-2012
In this pathch only the EM implemenation and the realisation of value profiler.
Thanks to Mikhail Fursov for his Edge profiler and for His Great help
to me. Thank You, Mikhail. =)

Re: [drlvm][jit] Seems like too many classes loaded

2006-10-30 Thread Alexey Varlamov


Just to finalize the issue: BEA's runtime also provides compilation
info, and it encountered 1081 methods on HelloWorldApp. Startup is
really costly :(

PS. I'm catching up after a week offline, sorry for resurrecting such
an old thread :$

24 Oct 2006 13:31:16 +0700, Egor Pasko <[EMAIL PROTECTED]>:

On the 0x20B day of Apache Harmony Armand Navabi wrote:
> I am trying to become more familiar with the jit code.  I ran the
> following to see what all was compiled when running Hello World.
> "java -Xtrace:compile Hello".  I was very surprised to see the number of
> methods that seem to be loaded.  I think there are about 1079 methods
> that get compiled for Hello World.
>
> Does harmony just load all the classes in the classlib and compile every
> method?  If not, then why would there be so many methods compiled for a
> simple hello world program?

my 2c:
first time methods are compiled "just before execution". So, you have
to execute a method in your app to see it compiled.

> Right now I am trying to write a simple profiling tool that counts the
> number of certain instructions in a program for a given input (i.e. I
> want to run the program, and for every instruction that ends up running,
> collect information).  So, since it seems that the jit not only compiles
> every instruction in the program (not only the ones that end up
> running), but also a bunch of other classes, I thought perhaps I would
> be more interested in the interpreter.

Do you mean bytecode instructions? or CPU instructions?  For bytecode
you better use the interpreter. For IA-32 we have a special profiling
utility in JIT CG ("iprof") that allows to count how many instructions
of a certain kind were executed, what are the hottest basicblocks,
etc. Feel free to ask on "iprof" in the mailing list. (I think, "we
should document this" :P, but it is not the first priority)

> Running "java -Xint -Xtrace:interpreter Hello" then also gave me way
> more output then I expected.  For example
>
> ...
> interpreter: java/lang/String indexOf(II)I
> interpreter: java/lang/String startsWith(Ljava/lang/String;)Z
> interpreter: java/lang/String startsWith(Ljava/lang/String;I)Z
> interpreter: java/lang/String regionMatches(ILjava/lang/String;II)Z
> interpreter: java/net/URI
> access$1002(Ljava/net/URI;Ljava/lang/String;)Ljava/lang/String;
> interpreter: java/net/URI access$1000(Ljava/net/URI;)Ljava/lang/String;
> interpreter: java/net/URI$Helper
> validatePath(Ljava/lang/String;Ljava/lang/String;I)V
> interpreter: java/net/URIEncoderDecoder
> validate(Ljava/lang/String;Ljava/lang/String;)V
> interpreter: java/lang/String length()I
> interpreter: java/lang/String charAt(I)C
> interpreter: java/lang/String indexOf(I)I
> interpreter: java/lang/String indexOf(II)I
> interpreter: java/lang/String length()I
> ...
>
> I'm not sure why all of this is being interpreted for a simple hello
> world program.  My understanding was that the interpreter traverses the
> byte code for the given input and then handles the executed byte codes.
> It seems like a lot more is going on here.
>
> Also, I am trying to become familiar with the jit and interpreter by
> reading the DRL Developer's Guide.  Are there any other resources other
> than the Developer's Guide and this mailing list?
>
>
> Thanks
> Armand
>

--
Egor Pasko, Intel Managed Runtime Division

Re: [drlvm][em] Value-profiling implementation is avaible.

2006-10-30 Thread Pavel Ozhdikhin

Hello Yuri,

I'm currently condidering different ways of devirtualization improvement in
the JIT compiler. There are many of them and value profiling is one of the
most promising. I would be happy if we could work together on making value
profiling useful in Jitrino.OPT. Currently devirtualizer in JIT can use edge
profile and method hotness profile (in addition to class hierarchy
analysis and preexistence) to guess about receiver type but value
profile must be more precise.

I'd like to look into your patch and play a little with it - could you
please put your code into JIRA and announce its number here?

Thank you,
Pavel Ozhdikhin
Intel Java and XML division

On 10/31/06, Yuri Kashnikoff <[EMAIL PROTECTED]> wrote:

2006/10/30, Mikhail Fursov <[EMAIL PROTECTED]>:
> Hi Yuri.
> Value profile is a profile that is really needed for set of JIT
> optimizations.
> What do you want to improve in JIT exactly?
First of all I want to imporove the devirtualization, using value
profiling feedback. And now I found that some other optimization could
use value profling, if it is interesting I can put some ideas here for
considering.

For Geir and other non-Jit-nerds (I'am not JIT-nerd alredy but I'am
trying to become one of them), I'll try to explain what the value
profiling is and how it could be used for increasing JIT performance.

Devirtualization of virtual calls.
We have some virtual calls. And we know that in some program
realisation of some method A used more often (for examle 1000 times)
and antoher realisation of this method B used only 10-20 times. So
this information could be usefull for recompiling the Hot Method A.
And Value Profiler brings this information to JIT.

Re: [drlvm][em] Value-profiling implementation is avaible.


2006/10/30, Mikhail Fursov <[EMAIL PROTECTED]>:

Hi Yuri.
Value profile is a profile that is really needed for set of JIT
optimizations.
What do you want to improve in JIT exactly?

First of all I want to imporove the devirtualization, using value
profiling feedback. And now I found that some other optimization could
use value profling, if it is interesting I can put some ideas here for
considering.

For Geir and other non-Jit-nerds (I'am not JIT-nerd alredy but I'am
trying to become one of them), I'll try to explain what the value
profiling is and how it could be used for increasing JIT performance.

Devirtualization of virtual calls.
We have some virtual calls. And we know that in some program
realisation of some method A used more often (for examle 1000 times)
and antoher realisation of this method B used only 10-20 times. So
this information could be usefull for recompiling the Hot Method A.
And Value Profiler brings this information to JIT.

Re: [drlvm] Class unloading support

2006-10-30 Thread Aleksey Ignatenko

Hello, Etienne.

Am I wrong, or does this proposition imply collecting classes
independently from their class loader?  If this is the case, I have to
say that I disagree with the proposed approach.

Yes, you are wrong. This proposition implies collection of classloader and
clasess loaded by it at once. You can see what is "class registry" in
the first letter of the discussion -

"Class registry - introduce references from j.l.Classes to its defining
j.l.Classloader and references from j.l.Classloader to j.l.Classes loaded by
it (unloading is to be done for j.l.Classloader and corresponding
j.l.Classes at once)."

And what about gagnon-phd.pdf:

very effective approach for managing class-loader related memory

Drlvm already has similar functionality: look at classloader.h, function
void* Alloc(size_t size); You'll see that most of classloader's data (not
100% yet) is already allocated from pool of that classloader.

Aleksey.

On 10/30/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:

Hi Weldon,

Weldon Washburn wrote:
> I read section 3.2.3 (Class-Loader-Specific Memory) of gagnon-phd.pdf .
> Please tell me if the following is a correct interpretation.  You create
> a new memory manager that is uniquely associated with each new class
> loader.

Right.

>  All the C data structures associated with a class loader (classes,
> vtables, etc) are "malloc()ed" out of the associated memory manager.

[For those who have not read it...]

"malloc()ed" is a big word...  It is rather "simpleAlloc()ed", i.e.,
once allocated, you cannot free it (...or if you do, the "free-list"
manager is very minimal, performs no checks [you have to tell it how
much you are freeing] and no aggregation).  I do discuss this in the
Chapter, of course, and you can look at the implementation in SableVM.
[The SableVM trunk is under AL2.0 (unlike released versions)].

>  When
> the class loader becomes unreachable, then its associated memory manager
is
> deallocated which automatically frees all the associated C structs
> (classes, vtables, etc.)

Yep.

Etienne

--
Etienne M. Gagnon, Ph.D. http://www.info2.uqam.ca/~egagnon/
SableVM:   http://www.sablevm.org/
SableCC:   http://www.sablecc.org/

Re: [drlvm][em] Value-profiling implementation is avaible.




Tim Ellison wrote:

Geir Magnusson Jr. wrote:

can one of you JIT nerds describe for us non-JIT-nerds what value
profiling is?  :)


Maybe calling people nerds was a disincentive ;-)



I hope not - it's a compliment. :)


Value profiling means watching the actual values of variables or
expressions in a running program, typically in order to direct
optimizations that are value-specific.

Regards,
Tim

Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)




Tim Ellison wrote:

Geir Magnusson Jr. wrote:

Alexei Zakharov wrote:

At the same time I don't feel completely comfortable with the idea
of using preprocessor to separate J2SE sources from J2ME.

I'm not overjoyed either, but I can't think of many ways that allow
fairly clear readability without don't require tons of IDE-specific
tooling.  This is what bothers me about aspects - can you get a real
clue what's going to happen looking at the source code?


To be fair though, it's pretty hard to see what is going on in any large
software system without IDE support.  If you haven't looked at AspectJ
[1] for a while it's worth looking again -- they do a pretty cool job of
showing what is going to happen with aspects applied.


I can't find the plugins for IDEA :)

I know what you mean, but I can tend to reach for any of the main IDEs 
and work on most projects.  It's just when a project mandates an IDE 
that I get itchy.


Now, I know in our case that we may have to do that and I'm willing to 
completely go to the dark side and use Eclipse all the time (and aspectJ 
has support for more than one because of it's history... it didn't start 
at Eclipse...), but I'm hoping there are enough clever people around so 
we can get a broad set of tools...




I agree with your point that it's not going to be helpful to require
"tons of IDE-specific tooling".

[1] http://www.eclipse.org/aspectj/

Regards,
Tim

Re: [doc][drlvm] new docs added - JIT compiler

is there any chance you could put a zip of the images into the jira, 
rather than a  dozen or so images?  That's actually what turned me off 
from one of the other doc issues - I said to myself "oh, heck, I don't 
want to download each of those separate thingies... I just want a zip or 
tgz of them... easier to handle..."


So please? :)

geir

Morozova, Nadezhda wrote:
All, 


New documents have been added to HARMONY-2003. These are a description
of the Jitrino.OPT and .JET compilers, with a supplementary doc on the
pipeline management framework inside the JITs. The docs describe the
architecture, processes and usage of the components.

 


It would be great if somebody took a look, expressed an opinion, and, if
favorable, committed to the website.

Your review or any other feedback are most welcome,

 

Thanks, 


Nadya Morozova

 


PS: this JIRA is not the only one unresolved doc issue. Find more useful
pending patches in our database :-)

Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks




Gregory Shimansky wrote:

On Tuesday 31 October 2006 02:27 Geir Magnusson Jr. wrote:

So I should either create a new 4th category for tests with custom build
file, or extend one of the current categories which we have. The most
close to what I need is probably smoke tests category. I need to add
building native code part and add a custom command line setting
somewhere.

Or do you think I should go with completely new tests category?

New category


Hmm... I shouldn't have asked, or I wouldn't receive two different answers :)

I don't want to create a huge discussion out of it like most [testing] 
discussions become. Just want to know your arguments to create one more tests 
category.


Because the current frameworks are... wacky.  I can't turn off smoke 
tests without *recompiling* the test.  The c-unit test rig is kinda 
cool, but inappropriate.  Maybe kernel could be used.


it sounds like you just want to launch a set of conventional junit tests 
with a special invocation of java to get the agent running, right?




Now that I looked at the smoke tests build more closely and found a paths 
problem which I don't know how to fix, I am also inclined to make my own 
build script to have it separated. I would at least know how it works 
and own this secret like someone who wrote smoke build script does.


That's what I was hoping to avoid.  Something conventional.  JUnit or 
TestNG (TestNG! TestNG!), and a separate ant script invoked from main 
script.


geir

Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks

Gregory Shimansky wrote:

On Tuesday 31 October 2006 00:24 Fedotov, Alexei A wrote:

Gregory wrote,

I need is probably smoke tests category. I need to add building native

code

part and add a custom command line setting somewhere.

+1
I believe you need one or two test with a good coverage to check your
changes regularly. This is enough for acceptance testing.

This doesn't inhibit the separate category - it would be quite useful
for thorough testing. But from my perspective this is not the first
thing to do.

Now if I just could understand the Grand Design behind that huge tangle called
drlvm ant build... I cannot find a place to start with. I thought eclipse ant
debugging facilities may help (after I read how to build classlib from
eclipse), and spent 2 hours trying to give it properties here and there to no
avail, building drlvm from eclipse didn't work for me so far.

That's why I said "new category". Do something conventional.

Anyway, I think I've found a path bug with running smoke tests. The paths
seems to be different depending on what I run "build.sh test" or "build.sh
smoke.test".

This is what I see when I run "build.sh smoke.test"

/home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java -Dvm.assert_dialog=0 -classpath /home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/interpreter/_smoke.tests/classes
StackTest

If you look closely you'll see that tests are taken from
vm/interpreter/_smoke.tests/classes while the correct location which is built
when I run "build.sh test" is vm/_smoke.tests/classes. There is no
interpreter path in it. If someone knows how to fix it, I'd be grateful.

Now that I've tried it a second time after a full rebuild the path looks like
this

Funny how it works picking up seemingly a random subdirectory in
build/lnx_ia32_gcc_debug/semis.

Re: [DRLVM][GC] parallel compaction and wasted virtual space


On 10/31/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:

Are IBM's and Sun's are know to work well for production loads?


Don't know. Probably yes. It's claimed that SUN's work is the first
parallel compaction design [1] and IBM's work is an improvement [2],
while Compressor is a further improvement [3].

Well anyway, too choose a high-level design is actually not a big
deal. I think the real issues come from details in implementation
(load balance and scalability). So probably I can write two parallel
compactors based on [2] and [3], and hopefully find some solution
suitable for Harmony with targeted workloads.

Thanks,
xiaofeng

[1] Christine Flood, Dave Detlefs, Nir Shavit, and Catherine Zhang.
Parallel garbage collection for shared memory multiprocessors. In
Usenix Java Virtual Machine Research and Technology Symposium
(JVM'01), Monterey, CA, April 2001.
[2] Diab Abuaiadh, Yoav Ossia, Erez Petrank, and Uri Silbershtein. An
efficient parallel heap compaction algorithm. In ACM Conference on
Object-Oriented Programming, Systems, Languages, and Applications
(OOPSLA'04), ACM SIGPLAN Notices, Vancouver, October 2004. ACM Press.
[3] Haim Kermany, Erez Petrank. The Compressor: concurrent,
incremental, and parallel compaction. In Proceedings of the 2006 ACM
SIGPLAN Conference on Programming Languages Design and Implementation
(PLDI 2006), Pages 354-363, 2006.


Xiao-Feng Li wrote:
> Weldon, the problem is, there is no well-established parallel
> compaction algorithms. So far the best known work are 1. SUN's work;
> 2. IBM's work and 3. Compressor. No one knows which one is the best
> for different workloads. We have to identify one algorithm for
> implementation, and at the moment Compressor looks to be the right
> choice, or we write more than one compactors.
>
> Thanks,
> xiaofeng
>
> On 10/30/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
>> Since the Compressor algorithm was published only this year, perhaps it
>> makes sense to consider it experimental.  Perhaps make it a compile time
>> switch so that that folks focused on production quality VM don't trip on
>> it.  Even assuming the implementation of the Compressor algorithm is bug
>> free, there may be unforeseen performance problems that surface with
>> different workloads.
>>
>> On 10/29/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>> >
>> > On 10/29/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
>> > > Xiao Feng,
>> > >   I will read the reference to understand what are the compressor
>> > > advantages, and how the algorithm is implemented, thanks.
>> > >
>> > > Even when you have 1GB of physical memory, is there not an
>> overhead of
>> > page
>> > > faults?
>> >
>> > Yes, I agree that page faults definitely will be an overhead. I guess
>> > the page mapping overhead in Compressor is lower than the benefits it
>> > achieves. But yes, we need evaluate it independently.
>> >
>> > > Is it an option to compact the heap in parts and/or to increase the
>> > number
>> > > of passes to reduce the space overhead?
>> >
>> > The key idea of Compressor is to keep the object order during parallel
>> > compaction. There are other algorithms like "mark-copy" which require
>> > less additional copy space, but can't maintain the object order. In
>> > order to enable the parallel compaction of multiple blocks, the
>> > assumption is, we have to assume the to-space in the worst case is the
>> > equal size as the from-space. We can use a to-space with 30% size of
>> > the from-space in most compaction collections without problem, but we
>> > need be prepared for the worst case. A possible solution is, to have a
>> > fall-back algorithm when the to-space is smaller than required. This
>> > is not a new idea, e.g., GCv4.1 employs something similar and there
>> > are some papers available. [1] in ISMM06 is an example.
>> >
>> > [1] http://www.cs.purdue.edu/homes/pmcgache/ismm06.pdf
>> >
>> > > Is this significantly better than doing semi-space copying at each GC
>> > cycle,
>> > > since one major advantage of compaction( other than preserving
>> > allocation
>> > > order ) over copying, was probably less space overhead?
>> >
>> > Yes. The major advantage in my opinion is less physical space
>> > overhead. Well it introduces the vitual space overhead. If assuming
>> > the same of physical space overhead as a semi-space collector, we need
>> > evaluate the real benefits of object locality to trade off the
>> > collection pause time.
>> >
>> > > Are we looking for a parallel compaction algorithm for all
>> situations,
>> > or
>> > > can we think of choosing at JVM startup based on user input,
>> > client/server,
>> > > or OS feedback on execution environment?
>> >
>> > I think some adaptive choice is better. It means we need provide the
>> > choices at first. :-) I guess it's not a big overhead to have two
>> > parallel compactors.
>> >
>> > Thanks,
>> > xiaofeng
>> >
>> > > Sorry for all these questions before reading the book :-)
>> > >
>> > > Rana
>> > >
>> > >
>> > >
>

Re: [DRLVM][GC] parallel compaction and wasted virtual space

On 10/30/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:

On 10/29/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>
> Weldon, the problem is, there is no well-established parallel
> compaction algorithms. So far the best known work are 1. SUN's work;
> 2. IBM's work and 3. Compressor. No one knows which one is the best
> for different workloads. We have to identify one algorithm for
> implementation, and at the moment Compressor looks to be the right
> choice, or we write more than one compactors.

Let me ask a different question.  At the beginning of the email chain you
said:
>If we have 1GB
>physical memory, the JVM is ok for Compressor because the virtual
>space is large enough to wast half; but if the phsical memory is >2GB,
>Compressor may have a problem in 32bit machine: some of phsical mapped
>space might be wasted.

Perhaps you mean to say that with the Compressor algorithm, the JVM's heap
is unable to grow beyond 2GB of virtual space.  The reason is because
Compressor double maps physical memory and thus can not grow beyond one half
of the virtual address space.  A 2GB Java heap can certainly run on 512MB of
physical memory.  The result will be unacceptable paging.   However, even if
a machine contains 4GB of physical memory, the Compressor still can not take
advantage of anything above 2GB of physical memory.

One question to ask is if restricting the size of the heap to less than 2GC
will meet the requirements for Harmony VM in the next 6 months or so.  If
the answer is no, an additional algorithm will need to be implemented.

Well I guess no one has the answer. :-)

Also,  the Comressor algorithm is only recently published.  Given that the
focus of Harmony is production quality JVM, there is a risk when
implementing any algorithm that is yet unproven in a production environment.

Thanks,
> xiaofeng
>
> On 10/30/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> > Since the Compressor algorithm was published only this year, perhaps it
> > makes sense to consider it experimental.  Perhaps make it a compile time
> > switch so that that folks focused on production quality VM don't trip on
> > it.  Even assuming the implementation of the Compressor algorithm is bug
> > free, there may be unforeseen performance problems that surface with
> > different workloads.
> >
>
> --
> Weldon Washburn
> Intel Enterprise Solutions Software Division

Re: [drlvm][sablevm] Desing of Class Unloading Support


On 10/31/06, Pavel Pervov <[EMAIL PROTECTED]> wrote:

Ignatenko vs Gagnon proposal checklist follows. :)


> In accordance with Section 2.17.8 of the JVM spec, class unloading (and
> its related native resource cleanup) can only happen when the class
> loader instance becomes unreachable.  For this to happen, we put in
> place the following things:

1- Each class loader is represented by some VM internal structure.
> [We'll call it the "class loader structure"].


This is true.



> 2- Each class loader internal structure, except (optionally) the
> bootstrap class loader, maintains a weak reference to an object
> instance of class ClassLoader (or some subclass).  The Java instance
> has some opaque pointer back to the internal VM structure.   The Java
> instance is usually created before the internal VM structure.  The
> instance constructor is usually in charge of creating the internal VM
> structure.  [We'll call it the "class loader instance"]


This is true.



> 3- Each class loader instance maintains a collection of loaded classes.
> A class/interface is never removed from this collection.  This
> collection maintains "hard" (i.e. "not weak") references to
> classes/interfaces.


This is true.



> 4- [Informative] A class loader instance is also most likely to maintain
> a collection of classes for which it has "initiated" class loading.
> This collection should use hard references (as weak references won't
> lead to earlier class loading).


This is not true. Look for the thread "[drlvm] Non-bug difference
HARMONY-1688?", where Eugene Ostrovsky desribed initiating loaders in
details with links to specification.


> 5- Each class loader instance maintains a hard reference to its parent
> class loader.  This reference is (optionally) null if the parent is the
> bootstrap class loader.


This is true. This is actually a part of delegation framework.



> 6- Each j.l.Class instance maintains a hard reference to the class
> loader instance of the class loader that has loaded it.  [This is not
> the "initiating" loaders, but really the "loading" loader].


This is true. AFAIU, this class loader is called "defining" loader for a
class.



> 7- Each class loader structure maintains a set of boolean flags, one
> flag per "non-nursery" garbage collected area (even when thread-local
> heaps are used).  The flag is set when an instance of a class loaded by
> this class leader is moved into the related GC-area.  The flag is unset
> when the GC-area is emptied, or (optionally) when it can be determined
> that no instance of a class loaded by this class loader remains in the
> GC-area.  This is best implemented as follows: a) use an unconditional
> write of "true" in the flag every time an object is moved into the
> GC-area by the garbage collector, b) unset the related flag in "all"
> class loader structures just before collecting a GC-area, then setting
> the flag back when an object survives in the area.


Requires identification of object' class type during GC. Will most
probably degrade GC performance.


Yes, this is also my concern.

Thanks,
xiaofeng


> 8- Each method invocation frame maintains a hard reference to either its
> surrounding instance (in case of instance methods, i.e. (invokevirtual,
> invokeinterface, and invokespecial) or its surrounding class
> (invokestatic).  This is already required for synchronized methods
> (it's not a good idea to allow the instance to be collected before the
> end of a synchronized instance method call; yep, learned the hard way
> in SableVM...)  So, the "overhead" is quite minimal.  The importance of
> this is in the correctness of not letting a class loader to die while a
> static/instance method of a class loaded by it is still active, leading
> to premature release of native resources (such as jitted code, etc.).


Not generally true for optimizing JITs. "This" (or "class") can be omitted
from enumeration if it is not used anywhere in the code. Generally, this
technique reduces number of registers used in the code ("register pressure"
they call it :)).



> 9- A little magic is required to prevent premature collection of a class
> loader instance and its loaded j.l.Class instances (see [3-] above), as
> object instances do not maintain a hard reference to their j.l.Class
> instance, yet we want to preserve the correctness of Object.getClass().
>
> So, the simplest approach is to maintain a hard reference in a class
> loader structure to its class loader instance (in addition to the weak
> reference in [2-] above).  This reference is kept always set (thus
> preventing collection of the class loader instance), except when *all*
> the following conditions are met:
> a) All nurseries are empty.
> b) All GC-area flags are unset.


This requires more involvment of a GC in unloading process and affects GC
code more. In DRLVM, GC is designed to be a replaceable component. Moreover,
we already have 3 different working GCs and MMTk on the way. So, including
GC into the d

Re: [drlvm][sablevm] Desing of Class Unloading Support


Etienne,
 This is a good design, thanks. Conceptually, reference counting in the VM
is somewhat similar to Aleksey's proposal 1, if I understand correctly. This
design also requires quite a few hand-offs between the VM and GC. In DRLVM,
the problem is that we have quite a few GC's, not all within our control.
 However, it seems to me that we can either desire to make unloading
automatic, in which case, we will need things like java vtables etc and
leave most things to the GC. Or we can do refcounting or tracing in the VM,
and work lock step with the GC(s). I am not sure which is the better way.

Thanks,
Rana


On 10/30/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:


Hi all,

Here's a more structured proposal for a simple and effective
implementation of class unloading support.

In accordance with Section 2.17.8 of the JVM spec, class unloading (and
its related native resource cleanup) can only happen when the class
loader instance becomes unreachable.  For this to happen, we put in
place the following things:

1- Each class loader is represented by some VM internal structure.
[We'll call it the "class loader structure"].

2- Each class loader internal structure, except (optionally) the
bootstrap class loader, maintains a weak reference to an object
instance of class ClassLoader (or some subclass).  The Java instance
has some opaque pointer back to the internal VM structure.   The Java
instance is usually created before the internal VM structure.  The
instance constructor is usually in charge of creating the internal VM
structure.  [We'll call it the "class loader instance"]

3- Each class loader instance maintains a collection of loaded classes.
A class/interface is never removed from this collection.  This
collection maintains "hard" (i.e. "not weak") references to
classes/interfaces.

4- [Informative] A class loader instance is also most likely to maintain
a collection of classes for which it has "initiated" class loading.
This collection should use hard references (as weak references won't
lead to earlier class loading).

5- Each class loader instance maintains a hard reference to its parent
class loader.  This reference is (optionally) null if the parent is the
bootstrap class loader.

6- Each j.l.Class instance maintains a hard reference to the class
loader instance of the class loader that has loaded it.  [This is not
the "initiating" loaders, but really the "loading" loader].

7- Each class loader structure maintains a set of boolean flags, one
flag per "non-nursery" garbage collected area (even when thread-local
heaps are used).  The flag is set when an instance of a class loaded by
this class leader is moved into the related GC-area.  The flag is unset
when the GC-area is emptied, or (optionally) when it can be determined
that no instance of a class loaded by this class loader remains in the
GC-area.  This is best implemented as follows: a) use an unconditional
write of "true" in the flag every time an object is moved into the
GC-area by the garbage collector, b) unset the related flag in "all"
class loader structures just before collecting a GC-area, then setting
the flag back when an object survives in the area.

8- Each method invocation frame maintains a hard reference to either its
surrounding instance (in case of instance methods, i.e. (invokevirtual,
invokeinterface, and invokespecial) or its surrounding class
(invokestatic).  This is already required for synchronized methods
(it's not a good idea to allow the instance to be collected before the
end of a synchronized instance method call; yep, learned the hard way
in SableVM...)  So, the "overhead" is quite minimal.  The importance of
this is in the correctness of not letting a class loader to die while a
static/instance method of a class loaded by it is still active, leading
to premature release of native resources (such as jitted code, etc.).

9- A little magic is required to prevent premature collection of a class
loader instance and its loaded j.l.Class instances (see [3-] above), as
object instances do not maintain a hard reference to their j.l.Class
instance, yet we want to preserve the correctness of Object.getClass().

So, the simplest approach is to maintain a hard reference in a class
loader structure to its class loader instance (in addition to the weak
reference in [2-] above).  This reference is kept always set (thus
preventing collection of the class loader instance), except when *all*
the following conditions are met:
a) All nurseries are empty.
b) All GC-area flags are unset.

Actually, for making this practical and preserving correctness, it's a
little trickier.  It requires a 2-step process, much like the
object-finalization dance.  Here's a typical example:

On a major collection, where all nurseries are collected, and some (but
not necessary all) other GC-areas are collected, we do the following
sequence of actions:
a) All class loader structures are visited.  All flags related to
  non-nursery GC-areas that we intend t

Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks

2006-10-30 Thread Gregory Shimansky

On Tuesday 31 October 2006 02:27 Geir Magnusson Jr. wrote:
> > So I should either create a new 4th category for tests with custom build
> > file, or extend one of the current categories which we have. The most
> > close to what I need is probably smoke tests category. I need to add
> > building native code part and add a custom command line setting
> > somewhere.
> >
> > Or do you think I should go with completely new tests category?
>
> New category

Hmm... I shouldn't have asked, or I wouldn't receive two different answers :)

I don't want to create a huge discussion out of it like most [testing] 
discussions become. Just want to know your arguments to create one more tests 
category.

Now that I looked at the smoke tests build more closely and found a paths 
problem which I don't know how to fix, I am also inclined to make my own 
build script to have it separated. I would at least know how it works 
and own this secret like someone who wrote smoke build script does.

-- 
Gregory Shimansky, Intel Middleware Products Division

Re: [drlvm] what's the difference between the structure VM_thread and HyThread?

On 10/30/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:

Just my $0.02:
Actually VM_thread contains thread-local data. The VM_thread is a thread
local data by itself. HyThread keeps a pointer to it, so VM_thread is always
accessed from HyThread.
Some of the components, e.g. GCv4 and GCv5, keeps all thread local data in
VM_thread, but others JIT, GC_CC use more up to date method and request TLS
slots directly from TM.

Today we are in a transition from the old model when all TLS data was stored
in VM_thread to the new model - when every component requests TLS slots
independently from TM

Agree with Nikolay and Mikhail, VM_Thread was used for all the thread
specific data of a JVM thread. The transition is to make VM_Thread
only for the threading module, one of all the modules that share
HyThread.

Thanks,
xiaofeng

On 10/30/06, Nikolay Kuznetsov <[EMAIL PROTECTED]> wrote:
>
> Yes, this data is non threading and separate fields of VM_thread
> structure should be replaced with independent values (data structures)
> stored in TLS under separate keys.
>
> Nik.
>

--
Mikhail Fursov

Re: [general] creation of "jdktools"

Ilya Neverov wrote:

Hello,

I want to gather opinions about structure of the "jdktools" component.

I'm going to create scripts for moving tools' sources from classlib/
to top-level directory jdktools/ and to prepare patches for build
system for building tools from new place.

Cool

I think the following structure will be appropriate for future
evolution of the jdktools:

jdktools/trunk/
  build.xml
  make/

Can we stop persisting this mistake?  Please call it "build" :)

  doc/
  modules/
  jre/ #  keytool, tool launcher go here
 build.xml #  classes go to jdk/jre/lib/tools.jar
 make/
 src/
  jdk/ #  javac, jarsigner go here
 build.xml #  classes go to jdk/lib/tools.jar
 make/
 src/
  jdwp/#  separate module for large component
 build.xml
 make/
 src/

Only comment is that we might want to pull the launcher out to be a 
peer.  Otherwise, I like it.

Assumptions which look reasonable for jdktool's build subsystem:

1) it works in presence of built classlib (as HDK binaries or as a
result of classlib phase of overall build);

yes - think of the same trick we do w/ DRLVM to "reach over" to find it. 
 I'd imagine the federated build to then have :

   trunk/
  working_classlib/
  working_vm/
  working_jdktools/

2) the 'jre' module is always built before building 'jdk' to provide
generic tool launcher and the jre/lib/tools.jar. Probably it will be
easy to obtain these items from HDK.

That's one reason why I'd pull the launcher out to it's own module

I'm rather newbie in the Harmony build system so your thoughts will be
very helpful.

Ant and make will be your friends here :)  Note that you will have 
native issues (because of the launcher), so please track the way that 
classlib does this wrt platforms to start, and if you find things that 
work better, suggest it.  Mark and Ollie are wizards here.

I'd suggest starting out to accommodate (windows,linux) X (x86, x86_64) 
if you grok what I mean, and do it in a way that it will be trivial to 
add other OSs or processor architectures (IPF, for example).  This might 
be a good place to figure out how this should work going forward for 
harmony, rather than experimenting in classlib.

Thank you

No, thank you :)

geir

-Ilya

On 10/19/06, Ilya Neverov <[EMAIL PROTECTED]> wrote:

Hi Geir,

Looks like that creating the "jdktools" source tree and build was
shaded by other tasks. I can help with preparing and checking updates
in the build system. Please let me know what needs to do in this area
(besides svn commits) to complete the task.

I'm especially interested in completing the move to "jdktools"
structure since there will be a home for the JDWP code, which has beed
voted but still resides in JIRA. Working with SVN will be easier.

Thanks.
-Ilya

On 10/4/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
> yep, that's the plan.   And once we have that, we can simplify the
> launcher as well...
>
> Tim Ellison wrote:
> > +1 for creating a jdktools directory.  The dependency on the classlib
> > launcher should be relatively light if we go with a simple tools
> > launcher that rewrites the tool invocation into a generic launcher
> > invocation.  You may recall the idea was discussed a while ago.
> >
> > So, for example,
> >   jdk/bin/javac -source 1.5 -J-Xmx200M  FooBar
> > is rewritten to
> >   jdk/jre/bin/java -cp jdk/lib/tools.jar;jdk/lib/ecj.jar -Xmx200M
> > org.apache.harmony.tools.javac.Main -source 1.5 FooBar
> >
> > and so on.
> >
> > Regards,
> > Tim
> >
> > Geir Magnusson Jr. wrote:
> >> Geir Magnusson Jr. wrote:
> >>> Now that we have javac, javah, javap (if Tim votes ;) and 
keytool, I'd

> >>> like to organize these and add them to the next snapshot.
> >> My bad - the javap isn't being voted on yet.  I was thinking of 
the jdwp

> >> vote... sorry
> >>
> >>> So I propose adding a new top-level directory called "jdktools" 
(and

> >>> rename "tools" to "project_tools") and create a build target that -
> >>> with a  dependency on classlib for the launcher - creates the 
'stuff'

> >>> needed to fill into the JDK.
> >>>
> >>> Any comments?
> >>>
> >>> geir
> >>>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

--
Thank you.
Ilya Neverov,
Intel Middleware Products Division

Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks

2006-10-30 Thread Gregory Shimansky

On Tuesday 31 October 2006 00:24 Fedotov, Alexei A wrote:
> Gregory wrote,
>
> >I need is probably smoke tests category. I need to add building native
>
> code
>
> >part and add a custom command line setting somewhere.
>
> +1
> I believe you need one or two test with a good coverage to check your
> changes regularly. This is enough for acceptance testing.
>
> This doesn't inhibit the separate category - it would be quite useful
> for thorough testing. But from my perspective this is not the first
> thing to do.

Now if I just could understand the Grand Design behind that huge tangle called 
drlvm ant build... I cannot find a place to start with. I thought eclipse ant 
debugging facilities may help (after I read how to build classlib from 
eclipse), and spent 2 hours trying to give it properties here and there to no 
avail, building drlvm from eclipse didn't work for me so far.

Anyway, I think I've found a path bug with running smoke tests. The paths 
seems to be different depending on what I run "build.sh test" or "build.sh 
smoke.test".

This is what I see when I run "build.sh smoke.test"

/home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java
 -Dvm.assert_dialog=0 -classpath 
/home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/interpreter/_smoke.tests/classes

StackTest

If you look closely you'll see that tests are taken from 
vm/interpreter/_smoke.tests/classes while the correct location which is built 
when I run "build.sh test" is vm/_smoke.tests/classes. There is no 
interpreter path in it. If someone knows how to fix it, I'd be grateful.

Now that I've tried it a second time after a full rebuild the path looks like 
this

/home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/deploy/jre/bin/java
 -Dvm.assert_dialog=0 -classpath 
/home/gregory/work/Harmony/harmony/enhanced/drlvm/trunk/build/lnx_ia32_gcc_debug/semis/vm/gc_gen/_smoke.tests/classes

StackTest

Funny how it works picking up seemingly a random subdirectory in 
build/lnx_ia32_gcc_debug/semis.

-- 
Gregory Shimansky, Intel Middleware Products Division

Re: [classlib][luni] signalis interruptus in hysock



Fedotov, Alexei A wrote:

Geir, All,

I have examined class library code. It seems that the solution we
invented (return EINTR, then loop) was always in place. :-)

Few comments on understanding:
1. EINTR (=4) is renamed to HYPORT_ERROR_SOCKET_INTERRUPTED (=-9).


Yes, I did that in one place to have it fit into the portlib error code 
set.  Someone may have done it in another.



2. The loop is coded by means of "goto select".


In code that calls the wrapper for the lowest-level select(), right?


3. The same pattern is dupdupduplicated several times.


That's another issue entirely :)



I have not examined all places, though there could be paths which do not
fit the pattern. Honestly, I have examined the only path:

pollSelectRead() ->
hysock_select_read() ->
hysock_select()

Summary:
We can keep this issue open or close  it as won't fix. Meanwhile we
should look for the real problem.


I don't understand this - what do you see as the real problem? 
Interruption from select due to signals is a fact of life under linux.


geir



With best regards,
Alexei Fedotov,
Intel Java & XML Engineering


-Original Message-
From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]
Sent: Thursday, October 26, 2006 6:21 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [classlib][luni] signalis interruptus in hysock



Fedotov, Alexei A wrote:

Geir,

Do I understand correctly that you suggest the following?

1. hysock_select as its name says should mimic a behavior of select,

i.

e. return the error code from select without changing it. It's ok to
print a rare debug message.

Yes, that's what I had the other do (and no, I see no reason to print a
debug message, as upper layers can print if they find an EINTR)


2. The correct place for the loop is the module where hysock_select

is

called, or, let me be precise, class lib guys are to fix our

networking

code.

My plan is to fix it as fixed the other one.  It turns out that there
are several layers between java and the OS...

geir



With best regards,
Alexei Fedotov,
Intel Java & XML Engineering


-Original Message-
From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 25, 2006 10:01 AM
To: harmony-dev@incubator.apache.org
Subject: Re: [classlib][luni] signalis interruptus in hysock



Weldon Washburn wrote:

It seems JIRA is down for maintenance.  If HARMONY-1904 is still

open

perhaps it makes sense to put a counter in the while (...) {

select...}

loop. And after every N loops, print a warning/diagnostic message.

For whom and to what end?  Why not just return EINTR (in hysock

speak)?

The
value for N would have to be tuned.  I don't know what the best

number

would
be. Given that 1904 patch is not the final solution, at least a

diagnostic

that hints at where the system hangs would be useful.  It might

make

sense

to even print a stack trace.   Also, I agree with Ivan below.

Signals

bugs

are very hard to debug.  And diagnostics can help us all understand

the

corner cases better.

But so far, no one has shown that the system hangs, or can hang,

simply

because we return EINTR

geir


On 10/20/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:

On 10/20/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:

Ivan Volosyuk wrote:

Well, I think that the solution is what Geir suggests. One think

which

bothers me is following. EINTR can happen in different places

and

the

situations can be quite rare in some circumstances. It can lead

to

hard to reproduce stability bugs (race conditions).

Can you give an example?

Half a year ago, I was working on the problem. Socket operations

get

sometimes interrupted. We have found out that it occurs sometime

after

GC. It was not quite easy as the application was quite big and
situation - quite rare.

Given the fact, that current implementation of monitor reservation
code can stop other thread in quite random fashion we should have

rock

solid support of EINTR handling everywhere the select(), poll()

calls

is used.

--
Ivan
Intel Enterprise Solutions Software Division


We should find a
way how to test the implementation.

+1!

:)

geir

-

Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail:

[EMAIL PROTECTED]

For additional commands, e-mail:

[EMAIL PROTECTED]

Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks




Gregory Shimansky wrote:

Hello

JVMTI implementation is quite a big piece of drlvm code which doesn't have any 
tests that are ran regularly. This may create regressions in JVMTI 
implementation which won't be caught early. So I want to add several small 
tests for JVMTI which would cover most of the currently implemented 
functionality.


Yay!



The specific of such tests is that they have to have a custom command line, to 
specify -agentlib: library. They also require to build native 
code. They need to be run in default (JIT) mode and on interpreter.


Ok.  You can do that from ant



Current tests which we have for commit checks for drlvm don't have these 
features out of the box. Cunit tests build native code, but don't run java 
executable. Smoke and kernel tests don't have native code and don't have a 
custom command line.


So I should either create a new 4th category for tests with custom build file, 
or extend one of the current categories which we have. The most close to what 
I need is probably smoke tests category. I need to add building native code 
part and add a custom command line setting somewhere.


Or do you think I should go with completely new tests category?


New category

geir

Re: [drlvm][em] Value-profiling implementation is avaible.

Geir Magnusson Jr. wrote:
> can one of you JIT nerds describe for us non-JIT-nerds what value
> profiling is?  :)

Maybe calling people nerds was a disincentive ;-)

Value profiling means watching the actual values of variables or
expressions in a running program, typically in order to direct
optimizations that are value-specific.

Regards,
Tim

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Geir Magnusson Jr. wrote:
> Alexei Zakharov wrote:
>> At the same time I don't feel completely comfortable with the idea
>> of using preprocessor to separate J2SE sources from J2ME.
> 
> I'm not overjoyed either, but I can't think of many ways that allow
> fairly clear readability without don't require tons of IDE-specific
> tooling.  This is what bothers me about aspects - can you get a real
> clue what's going to happen looking at the source code?

To be fair though, it's pretty hard to see what is going on in any large
software system without IDE support.  If you haven't looked at AspectJ
[1] for a while it's worth looking again -- they do a pretty cool job of
showing what is going to happen with aspects applied.

I agree with your point that it's not going to be helpful to require
"tons of IDE-specific tooling".

[1] http://www.eclipse.org/aspectj/

Regards,
Tim

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [classlib] Preprocessor

Geir Magnusson Jr. wrote:
> Tim Ellison wrote:
>> Where you do go through a source-to-source transform stage it helps of
>> you can collaborate with the second-stage compiler to pass through
>> original line number info (a la JSR45) otherwise debugging gets a bit
>> tiresome.
> 
> Right, except I could imagine having an IDE plugin that deals with this
> for you - you edit and debug using the processed code, which is what the
> line numbers will correspond to...

:-) of course, when I say "you" I mean the tooling not the developer.

Regards,
Tim

-- 

Tim Ellison ([EMAIL PROTECTED])

RE: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks

Gregory wrote,
>I need is probably smoke tests category. I need to add building native
code
>part and add a custom command line setting somewhere.

+1
I believe you need one or two test with a good coverage to check your
changes regularly. This is enough for acceptance testing. 

This doesn't inhibit the separate category - it would be quite useful
for thorough testing. But from my perspective this is not the first
thing to do. 

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Gregory Shimansky [mailto:[EMAIL PROTECTED]
>Sent: Monday, October 30, 2006 11:25 PM
>To: harmony-dev@incubator.apache.org
>Subject: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm
commit
>checks
>
>Hello
>
>JVMTI implementation is quite a big piece of drlvm code which doesn't
have
>any
>tests that are ran regularly. This may create regressions in JVMTI
>implementation which won't be caught early. So I want to add several
small
>tests for JVMTI which would cover most of the currently implemented
>functionality.
>
>The specific of such tests is that they have to have a custom command
line,
>to
>specify -agentlib: library. They also require to build
native
>code. They need to be run in default (JIT) mode and on interpreter.
>
>Current tests which we have for commit checks for drlvm don't have
these
>features out of the box. Cunit tests build native code, but don't run
java
>executable. Smoke and kernel tests don't have native code and don't
have a
>custom command line.
>
>So I should either create a new 4th category for tests with custom
build
>file,
>or extend one of the current categories which we have. The most close
to
>what
>I need is probably smoke tests category. I need to add building native
code
>part and add a custom command line setting somewhere.
>
>Or do you think I should go with completely new tests category?
>
>--
>Gregory Shimansky, Intel Middleware Products Division

RE: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Alexey wrote,
> > At the same time I don't feel
>> completely comfortable with the idea of using preprocessor to
separate
>> J2SE sources from J2ME.

A preprocessor looks a possible choice for maintaining several profiles
(eg Java ME vs Java SE). We need compact and preferably precompiled
sources to be efficient on small devices.

There is one more option. Can we use a good source control system
instead? Java ME workspace could be a child of Java SE one. I believe
something like BitKeeper or Teemware can keep things different when
needed and propagate bug fixes and patches painlessly for the common
part.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Alexei Zakharov [mailto:[EMAIL PROTECTED]
>Sent: Monday, October 30, 2006 8:05 PM
>To: harmony-dev@incubator.apache.org
>Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
smell -
>Thread.sleep() in ActivationGroup method)
>
>Hi all,
>
>Well, as an individual who has the tendency to pure Java programming I
>will be happier if I can control things on the source-code level. I
>can't say I don't like the idea about sophisticated JIT with the
>powerful AI inside, but if we are talking about logging then IMHO a
>good preprocessor is the thing that we need (but we may also continue
>to JIT away stuff if we like). At the same time I don't feel
>completely comfortable with the idea of using preprocessor to separate
>J2SE sources from J2ME.
>
>No clue about particular technology. It can be SableCC, something
>custom-made, velocity or whatever.
>
>Thanks,
>
>2006/10/30, Fedotov, Alexei A <[EMAIL PROTECTED]>:
>>Premature optimization is the root of all evil
>>Donald Knuth
>>
>>
>> >Just a small idea: Let teach JIT to purge this code unless special
>> option
>> >is ON
>>
>> +1
>>
>> I believe a computer should adapt to my way of thinking. I prefer a
>> simple and readable code to an efficient one since the former one
saves
>> the time of my life, why the latter one saves some electricity.
>>
>> With best regards,
>> Alexei Fedotov,
>> Intel Java & XML Engineering
>>
>> >-Original Message-
>> >From: Mikhail Fursov [mailto:[EMAIL PROTECTED]
>> >Sent: Sunday, October 29, 2006 8:56 PM
>> >To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
>> >Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
>> smell -
>> >Thread.sleep() in ActivationGroup method)
>> >
>> >On 10/29/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>> >>
>> >>
>> >> 1) The Logging Debate That Won't Die - we don't want to encumber
our
>> >> "production" code with logging or even with runtime enablement
checks
>> >> for logging i.e.
>> >>
>> >>   if (logging.isDebugEnabled())
>> >>
>> >> but it's clear that some people still want to use it for
debugging.
>> >
>> >
>> >Just a small idea: Let teach JIT to purge this code unless special
>> option
>> >is ON
>> >? Doing this we solve performance issue at least .
>> >
>> >If we did this, I assume that our build becomes a two step process,
>> >> first pre-process the code to create  separate "buildable source",
>> which
>> >> would go into source jars and such for debugging purposes.  Then
our
>> >> current javac/jar process.
>> >>
>> >> I'd also like to be able to work in an IDE with the pre-proc stuff
>> >> invisible if possible...
>> >
>> >
>> >This is the main problem. Backporting of your changes from the
>> "buildable
>> >source" to the "source with preprocessor" could have more overhead
then
>> >support of a separate branch for different Java version.
>
>
>--
>Alexei Zakharov,
>Intel Enterprise Solutions Software Division

RE: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Geir wrote,
>Not only that, we create a class library that places **weird**
requirements
>on any VM that wants to use it.

Ok, I believe this was honest. :-) As far as I know the optimization
Mikhail is talking about is nearly standard for any optimizing JIT. 

Isn't it just a simple inlining? I accept that inlining in Java is a bit
more complex, than in C, since you can load classes and redefine methods
on fly. Anyway optimizing JIT recompiles the whole thing when such event
happens and an appropriate callback is called.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]
>Sent: Monday, October 30, 2006 9:29 PM
>To: harmony-dev@incubator.apache.org
>Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
smell -
>Thread.sleep() in ActivationGroup method)
>
>
>
>Alexei Zakharov wrote:
>> Hi all,
>>
>> Well, as an individual who has the tendency to pure Java programming
I
>> will be happier if I can control things on the source-code level. I
>> can't say I don't like the idea about sophisticated JIT with the
>> powerful AI inside, but if we are talking about logging then IMHO a
>> good preprocessor is the thing that we need (but we may also continue
>> to JIT away stuff if we like).
>
>Not only that, we create a class library that places weird requirements
>on any VM that wants to use it.  No thanks.
>
> > At the same time I don't feel
>> completely comfortable with the idea of using preprocessor to
separate
>> J2SE sources from J2ME.
>
>I'm not overjoyed either, but I can't think of many ways that allow
>fairly clear readability without don't require tons of IDE-specific
>tooling.  This is what bothers me about aspects - can you get a real
>clue what's going to happen looking at the source code?
>
>geir
>
>>
>> No clue about particular technology. It can be SableCC, something
>> custom-made, velocity or whatever.
>>
>> Thanks,
>>
>> 2006/10/30, Fedotov, Alexei A <[EMAIL PROTECTED]>:
>>>Premature optimization is the root of all
evil
>>>Donald Knuth
>>>
>>>
>>> >Just a small idea: Let teach JIT to purge this code unless special
>>> option
>>> >is ON
>>>
>>> +1
>>>
>>> I believe a computer should adapt to my way of thinking. I prefer a
>>> simple and readable code to an efficient one since the former one
saves
>>> the time of my life, why the latter one saves some electricity.
>>>
>>> With best regards,
>>> Alexei Fedotov,
>>> Intel Java & XML Engineering
>>>
>>> >-Original Message-
>>> >From: Mikhail Fursov [mailto:[EMAIL PROTECTED]
>>> >Sent: Sunday, October 29, 2006 8:56 PM
>>> >To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
>>> >Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
>>> smell -
>>> >Thread.sleep() in ActivationGroup method)
>>> >
>>> >On 10/29/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>> >>
>>> >>
>>> >> 1) The Logging Debate That Won't Die - we don't want to encumber
our
>>> >> "production" code with logging or even with runtime enablement
checks
>>> >> for logging i.e.
>>> >>
>>> >>   if (logging.isDebugEnabled())
>>> >>
>>> >> but it's clear that some people still want to use it for
debugging.
>>> >
>>> >
>>> >Just a small idea: Let teach JIT to purge this code unless special
>>> option
>>> >is ON
>>> >? Doing this we solve performance issue at least .
>>> >
>>> >If we did this, I assume that our build becomes a two step process,
>>> >> first pre-process the code to create  separate "buildable
source",
>>> which
>>> >> would go into source jars and such for debugging purposes.  Then
our
>>> >> current javac/jar process.
>>> >>
>>> >> I'd also like to be able to work in an IDE with the pre-proc
stuff
>>> >> invisible if possible...
>>> >
>>> >
>>> >This is the main problem. Backporting of your changes from the
>>> "buildable
>>> >source" to the "source with preprocessor" could have more overhead
then
>>> >support of a separate branch for different Java version.
>>
>>

235 tests are missed at DRLVM test run for Windows

Vladimir,

I compared http://harmonytest.org/testapp.do?method=showrun&id=72 and
http://harmonytest.org/testapp.do?method=showrun&id=80.

I've noticed that the following tests are absent in DRLVM test run. Do
you have any thoughts why they are missed? 

org.apache.harmony.luni.tests.java.lang.SystemTest
org.apache.harmony.luni.tests.java.lang.ThreadDeathTest
org.apache.harmony.luni.tests.java.lang.ThreadGroupTest
org.apache.harmony.luni.tests.java.lang.ThreadLocalTest
org.apache.harmony.luni.tests.java.lang.ThreadTest
org.apache.harmony.luni.tests.java.lang.ThrowableTest
org.apache.harmony.luni.tests.java.lang.TypeNotPresentExceptionTest
org.apache.harmony.luni.tests.java.lang.UnknownErrorTest
org.apache.harmony.luni.tests.java.lang.UnsatisfiedLinkErrorTest
org.apache.harmony.luni.tests.java.lang.UnsupportedOperationExceptionTes
t
org.apache.harmony.luni.tests.java.lang.VerifyErrorTest
org.apache.harmony.luni.tests.java.lang.VirtualMachineErrorTest
org.apache.harmony.luni.tests.java.net.HttpRetryExceptionTest
org.apache.harmony.luni.tests.java.util.AbstractCollectionTest
org.apache.harmony.luni.tests.java.util.ArrayListTest
org.apache.harmony.luni.tests.java.util.ArraysTest
org.apache.harmony.luni.tests.java.util.BitSetTest
org.apache.harmony.luni.tests.java.util.CollectionsTest
org.apache.harmony.luni.tests.java.util.DateTest
org.apache.harmony.luni.tests.java.util.DuplicateFormatFlagsExceptionTes
t
org.apache.harmony.luni.tests.java.util.FormatFlagsConversionMismatchExc
eptionTest
org.apache.harmony.luni.tests.java.util.FormatterClosedExceptionTest
org.apache.harmony.luni.tests.java.util.FormatterTest
org.apache.harmony.luni.tests.java.util.HashMapTest
org.apache.harmony.luni.tests.java.util.IdentityHashMapTest
org.apache.harmony.luni.tests.java.util.IllegalFormatCodePointExceptionT
est
org.apache.harmony.luni.tests.java.util.IllegalFormatConversionException
Test
org.apache.harmony.luni.tests.java.util.IllegalFormatFlagsExceptionTest
org.apache.harmony.luni.tests.java.util.IllegalFormatPrecisionExceptionT
est
org.apache.harmony.luni.tests.java.util.IllegalFormatWidthExceptionTest
org.apache.harmony.luni.tests.java.util.InputMismatchExceptionTest
org.apache.harmony.luni.tests.java.util.LocaleTest
org.apache.harmony.luni.tests.java.util.MissingFormatArgumentExceptionTe
st
org.apache.harmony.luni.tests.java.util.MissingFormatWidthExceptionTest
org.apache.harmony.luni.tests.java.util.UnknownFormatConversionException
Test
org.apache.harmony.luni.tests.java.util.UnknownFormatFlagsExceptionTest
org.apache.harmony.luni.tests.java.util.UUIDTest
org.apache.harmony.luni.tests.java.util.VectorTest
org.apache.harmony.luni.tests.java.util.WeakHashMapTest
org.apache.harmony.luni.tests.util.Base64Test
org.apache.harmony.luni.tests.util.MsgHelpTest
org.apache.harmony.luni.tests.util.NYITest
org.apache.harmony.nio_char.tests.java.nio.charset.CharacterCodingExcept
ionTest
org.apache.harmony.prefs.tests.java.util.prefs.PreferencesTest
org.apache.harmony.print.tests.javax.print.PrintExceptionTest
org.apache.harmony.rmi.AccessExceptionTest
org.apache.harmony.rmi.AlreadyBoundExceptionTest
org.apache.harmony.rmi.ServerExceptionTest
org.apache.harmony.rmi.server.LogStreamTest
org.apache.harmony.rmi.server.RMIClassLoaderTest
org.apache.harmony.rmi.ServerRuntimeExceptionTest
org.apache.harmony.rmi.StubNotFoundExceptionTest
org.apache.harmony.rmi.tests.java.rmi.activation.ActivateFailedException
Test
org.apache.harmony.rmi.tests.java.rmi.activation.ActivationExceptionTest
org.apache.harmony.rmi.tests.java.rmi.activation.ActivationGroupDesc_Com
mandEnvironmentTest
org.apache.harmony.rmi.tests.java.rmi.activation.ActivationGroupDescTest
org.apache.harmony.rmi.tests.java.rmi.activation.UnknownGroupExceptionTe
st
org.apache.harmony.rmi.tests.java.rmi.activation.UnknownObjectExceptionT
est
org.apache.harmony.rmi.UnexpectedExceptionTest
org.apache.harmony.rmi.UnknownHostExceptionTest
org.apache.harmony.sql.tests.javax.transaction.xa.XAExceptionTest
org.apache.harmony.sql.tests.javax.transaction.xa.XAResourceTest
org.apache.harmony.sql.tests.javax.transaction.xa.XidTest
org.apache.harmony.tests.internal.net.www.protocol.http.HttpURLConnectio
nTest
org.apache.harmony.tests.internal.net.www.protocol.https.HttpsURLConnect
ionTest
org.apache.harmony.tests.java.lang.EnumTest
org.apache.harmony.tests.java.lang.instrument.ClassDefinitionTest
org.apache.harmony.tests.java.math.BigIntegerXorTest
org.apache.harmony.tests.java.net.InetAddressTest
org.apache.harmony.tests.java.net.URITest
org.apache.harmony.tests.java.net.URLConnectionTest
org.apache.harmony.tests.java.net.URLEncoderTest
org.apache.harmony.tests.java.net.URLTest
org.apache.harmony.tests.java.util.regex.Matcher2Test
org.ietf.jgss.OidTest
tests.api.java.io.BufferedInputStreamTest
tests.api.java.io.BufferedOutputStreamTest
tests.api.java.io.BufferedReaderTest
tests.api.java.io.BufferedWriterTest
tests.api.java.io.ByteArrayInputStreamTest
tests.api.java.io.ByteArrayOutputStreamTest
tests

[drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks

2006-10-30 Thread Gregory Shimansky

Hello

JVMTI implementation is quite a big piece of drlvm code which doesn't have any 
tests that are ran regularly. This may create regressions in JVMTI 
implementation which won't be caught early. So I want to add several small 
tests for JVMTI which would cover most of the currently implemented 
functionality.

The specific of such tests is that they have to have a custom command line, to 
specify -agentlib: library. They also require to build native 
code. They need to be run in default (JIT) mode and on interpreter.

Current tests which we have for commit checks for drlvm don't have these 
features out of the box. Cunit tests build native code, but don't run java 
executable. Smoke and kernel tests don't have native code and don't have a 
custom command line.

So I should either create a new 4th category for tests with custom build file, 
or extend one of the current categories which we have. The most close to what 
I need is probably smoke tests category. I need to add building native code 
part and add a custom command line setting somewhere.

Or do you think I should go with completely new tests category?

-- 
Gregory Shimansky, Intel Middleware Products Division

Re: [build] Building on Eclipse - FYI

Sian January wrote:
My concern is that if there is too much documentation and it's not well 
structured then people won't be inclined to read any of it.

Agreed.

 If 50% 
of the "Getting Started With DRLVM" page is actually about Eclipse, my 
feeling is that it won't be as useful for people who actually want to 
get started with DRLVM than it would be if it was half the size and more 
concise.

Agreed.  It should be about IDEA anyway :)

 In general I think having Eclipse-related documentation on 
its own page (or inline where it's relevant) is a good thing, I was just 
commenting on the amount of seemingly unrelated Eclipse documentation 
that is currently included in the DRLVM page ( 
http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started.html).  
Just my 2p worth...

I agree that our basic "getting started with Harmony" stuff should be 
relatively independent of any IDE, but have links with material for 
specific IDE's if people contribute the material.

What I thought I was responding to was the idea that we we should limit 
the amount of Eclipse tutorial in our *document about Eclipse*.  Sorry 
if I misunderstood.

geir

Thanks,

Sian

On 30/10/06, *Geir Magnusson Jr.* <[EMAIL PROTECTED] 
> wrote:

Sian January wrote:
 > I'm actually not sure if that first page is up to date or not  (
 >

http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclipse.html).
 >
 > I think it should be but I have not ever been able to get it to work
 > myself.  Can anyone else confirm whether it is still in date or not?
 >
 > Also I'm just wondering whether we really need steps 5-14 of
"Running an
 > Application in Eclipse" or any of "Debugging an Application in
Eclipse"?
 > There doesn't seem to be anything Harmony-specific in those
steps, it's
 > just
 > about using Eclipse.  I know Eclipse itself contains that kind of
 > documentation in its help system so IMHO it seems like it could
be a little
 > redundant to duplicate that on the Harmony website.
 >

I think that if we have an eclipse document, and the info is accurate, I
say why not?

We're trying to suck people into the whirling vortex that is our little
project, and if it's someone that isn't an eclipse user, but was
intrigued by our ability to run it, etc, etc, it seem to be harmless.  I
assume that it won't change often, so the maintenance is minor?

geir

 > Thanks,
 >
 > Sian
 >
 >
 > On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]
> wrote:
 >>
 >> Sian,
 >>
 >> Thanks a lot for the detailed description of what I should do. I'll
 >> follow your recommendations. :)
 >>
 >> I'll investigate what we have already written about Eclipse to
avoid
 >> redundant info and repeating pages.
 >>
 >> What I've just come across are:
 >> 1) The page describing how to set up Eclipse to develop Java code in
 >> Apache Harmony
 >> [
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
 >> pse.html];
 >> 2) The sections "Running an Application in Eclipse" and
"Debugging an
 >> Application in Eclipse "in the Getting Started with DRL
 >>
[file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli

 >> pse_Hello_world]
 >>
 >> Would be great, if you find a chance to check whether the
aforementioned
 >> info is up-to-date :)
 >>
 >> Cheers,
 >> Sveta
 >> -Original Message-
 >> From: Sian January [mailto:[EMAIL PROTECTED]
]
 >> Sent: Friday, October 27, 2006 3:34 PM
 >> To: harmony-dev@incubator.apache.org

 >> Subject: Re: [build] Building on Eclipse - FYI
 >>
 >> Hi Sveta,
 >>
 >> That sounds like a great idea.  I think the two main things you
need to
 >> do
 >> extra on Eclipse on Windows are as follows:
 >>
 >> 1. Make a copy of vsvars32.bat from your Visual Studio install
 >> directory.
 >> If you have chosen the defaults when installing you will find it in
 >> "C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
 >> Copy it
 >> to somewhere convenient such as the desktop and add the
following line
 >> (just
 >> below the last line that begins "@set..."):
 >> start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
 >> (where ... is the path to your eclipse installation).  NB. using
 >> "-vmargs
 >> -Xmx512M" is optional, but helpful to stop Eclipse running out of
 >> memory.
 >> Now just double click on this file to start Eclipse.
 >>
 >> 2. After you have started Eclipse and checked out Harmony from
SVN, copy
 >> ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib an

Re: [DRLVM][GC] parallel compaction and wasted virtual space


Are IBM's and Sun's are know to work well for production loads?


Xiao-Feng Li wrote:

Weldon, the problem is, there is no well-established parallel
compaction algorithms. So far the best known work are 1. SUN's work;
2. IBM's work and 3. Compressor. No one knows which one is the best
for different workloads. We have to identify one algorithm for
implementation, and at the moment Compressor looks to be the right
choice, or we write more than one compactors.

Thanks,
xiaofeng

On 10/30/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:

Since the Compressor algorithm was published only this year, perhaps it
makes sense to consider it experimental.  Perhaps make it a compile time
switch so that that folks focused on production quality VM don't trip on
it.  Even assuming the implementation of the Compressor algorithm is bug
free, there may be unforeseen performance problems that surface with
different workloads.

On 10/29/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>
> On 10/29/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
> > Xiao Feng,
> >   I will read the reference to understand what are the compressor
> > advantages, and how the algorithm is implemented, thanks.
> >
> > Even when you have 1GB of physical memory, is there not an 
overhead of

> page
> > faults?
>
> Yes, I agree that page faults definitely will be an overhead. I guess
> the page mapping overhead in Compressor is lower than the benefits it
> achieves. But yes, we need evaluate it independently.
>
> > Is it an option to compact the heap in parts and/or to increase the
> number
> > of passes to reduce the space overhead?
>
> The key idea of Compressor is to keep the object order during parallel
> compaction. There are other algorithms like "mark-copy" which require
> less additional copy space, but can't maintain the object order. In
> order to enable the parallel compaction of multiple blocks, the
> assumption is, we have to assume the to-space in the worst case is the
> equal size as the from-space. We can use a to-space with 30% size of
> the from-space in most compaction collections without problem, but we
> need be prepared for the worst case. A possible solution is, to have a
> fall-back algorithm when the to-space is smaller than required. This
> is not a new idea, e.g., GCv4.1 employs something similar and there
> are some papers available. [1] in ISMM06 is an example.
>
> [1] http://www.cs.purdue.edu/homes/pmcgache/ismm06.pdf
>
> > Is this significantly better than doing semi-space copying at each GC
> cycle,
> > since one major advantage of compaction( other than preserving
> allocation
> > order ) over copying, was probably less space overhead?
>
> Yes. The major advantage in my opinion is less physical space
> overhead. Well it introduces the vitual space overhead. If assuming
> the same of physical space overhead as a semi-space collector, we need
> evaluate the real benefits of object locality to trade off the
> collection pause time.
>
> > Are we looking for a parallel compaction algorithm for all 
situations,

> or
> > can we think of choosing at JVM startup based on user input,
> client/server,
> > or OS feedback on execution environment?
>
> I think some adaptive choice is better. It means we need provide the
> choices at first. :-) I guess it's not a big overhead to have two
> parallel compactors.
>
> Thanks,
> xiaofeng
>
> > Sorry for all these questions before reading the book :-)
> >
> > Rana
> >
> >
> >
> >
> >
> >
> > > On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
> > > >
> > > > Hi, all, the plan for GCv5 parallel compaction is to apply the 
idea

> of
> > > > Compressor [1]. But it has an issue I want to discuss with you.
> > > > Compressor needs to reserve an unmapped virtual space for
> compaction.
> > > > The size of the reserved part is the same as that of copy reserve
> > > > space in a semi-space collector. This means about that part of 
the

> > > > virtual space is unusable for the JVM. In a typical setting, the
> > > > wasted part is half size of the total compaction space. If we 
have

> 1GB
> > > > physical memory, the JVM is ok for Compressor because the virtual
> > > > space is large enough to wast half; but if the phsical memory is
> >2GB,
> > > > Compressor may have a problem in 32bit machine: some of phsical
> mapped
> > > > space might be wasted.
> > > >
> > > > Any opinion on this?
> > > >
> > > > Thanks,
> > > > xiaofeng
> > > >
> > > > [1] http://www.cs.technion.ac.il/~erez/Papers/compressor-pldi.pdf
> > >
> > >
> >
> >
>



--
Weldon Washburn
Intel Enterprise Solutions Software Division

Re: [drlvm][sablevm] Desing of Class Unloading Support




Etienne Gagnon wrote:

[SNIP]



I hope this is useful to both projects [drlvm][sablevm]  :-)


This was really great - I need to go back and read it carefully.  Thanks
so much!



Etienne

(C) 2006 by Etienne M. Gagnon <[EMAIL PROTECTED]>
This text is licensed under the Apache License, Version 2.0.

[You may add this document in svn;  I am willing to sign the required
Apache agreement to make it so, if you intend to use it in drlvm's
implementation].


This isn't really necessary - by the terms of this list, anything
submitted is considered a contribution under the terms of the Apache
license and ICLA, unless noted otherwise as "NOT A CONTRIBUTION".

However, it would be great if you had an ICLA and ACQ on file to save
you the trouble of typing this in the future :)  "Better living through 
paperwork!"


geir

Re: [drlvm][em] Value-profiling implementation is avaible.

can one of you JIT nerds describe for us non-JIT-nerds what value 
profiling is?  :)


geir


Yuri Kashnikoff wrote:

Hi!
I'am working on value-profiling implementation. Implementation in EM
is alredy avaible. Now I'am trying to implement and use it in JIT. I
can answer any questions about current implementation. I have used the
straigth algorithm (cacthing FIRST N values) as default and as
advanced algorithm the Top-N-Value algorithm from the B. Calder, P.
Feller "Value Profiling and Optimisation". I'll appreciate any
questions. Please review and comment.

Re: [classlib] Preprocessor




Tim Ellison wrote:

Etienne Gagnon wrote:

Chris Gray wrote:

For JavaME I think it's the only way we'd be able to maintain a single
source tree. We need to be able to "#ifdef out" references to classes
we don't have, methods we don't implement, etc..

That much being said, I don't have a recommendation for a tool to use.
Java syntax is sufficiently C-like that cpp is probably do-able, but
we'd probably stumble over a whole series of gotcha's, and cpp isn't
exactly [deity]'s gift to preprocessing anyway. Maybe one of the
aspect-oriented tools (with which I am not at all familiar) could be a
better bet?

You could always do "clean" source-to-source processing using
SableCC...:-)  Java is a nice language to parse, so you could do some
clean parsing, instead of the dumb "unstructured text" replacement of
preprocessors.

Actually, if all you need if "ifdef'ing out" undesirable references, it
could be done by "hiding" modification directives in structured
comments,  so that these comment remain "javac" invisible.  This way you
could make it such that:
1- Plain source compilation -> j2se .
2- Structured processed source compilation -> j2me .


I agree, ensuring that the original source remains compilable can be a
great benefit.

Besides in-lining #ifdef's you can also maintain look-aside files with a
description of the smaller configurations.  That helps to avoid code
clutter though of course you may prefer to be marking-up the code
in-line if it is not simply removing whole types/methods.


Yes.



The other thing to remember is that methods that appear in the smaller
configurations must only be implemented in terms of methods that appear
in the smaller configurations.  For example, you may have to rewrite
regular IO to not be implemented in terms of NIO so that it remains
viable in configurations that don't have NIO.  In Harmony we have a
common 'platform IO layer' used for both modules.

Where you do go through a source-to-source transform stage it helps of
you can collaborate with the second-stage compiler to pass through
original line number info (a la JSR45) otherwise debugging gets a bit
tiresome.


Right, except I could imagine having an IDE plugin that deals with this 
for you - you edit and debug using the processed code, which is what the 
line numbers will correspond to...




Regards,
Tim


If you need it, there are 2 or 3 Java 1.5 grammars available for SableCC
(different parsing approaches, not different syntax!).  As I said, Java
is a pleasure to parse when compared to C & C++.

It's just an idea, of course...  [I know that people can start religious
wars about pre-processing; that's why I am suggesting a clean approach,
so that j2se people don't have to pre-process].

Etienne

Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Alexei Zakharov wrote:

Hi all,

Well, as an individual who has the tendency to pure Java programming I
will be happier if I can control things on the source-code level. I
can't say I don't like the idea about sophisticated JIT with the
powerful AI inside, but if we are talking about logging then IMHO a
good preprocessor is the thing that we need (but we may also continue
to JIT away stuff if we like). 

Not only that, we create a class library that places weird requirements 
on any VM that wants to use it.  No thanks.

> At the same time I don't feel

completely comfortable with the idea of using preprocessor to separate
J2SE sources from J2ME.

I'm not overjoyed either, but I can't think of many ways that allow 
fairly clear readability without don't require tons of IDE-specific 
tooling.  This is what bothers me about aspects - can you get a real 
clue what's going to happen looking at the source code?

geir

No clue about particular technology. It can be SableCC, something
custom-made, velocity or whatever.

Thanks,

2006/10/30, Fedotov, Alexei A <[EMAIL PROTECTED]>:

   Premature optimization is the root of all evil
   Donald Knuth

>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON

+1

I believe a computer should adapt to my way of thinking. I prefer a
simple and readable code to an efficient one since the former one saves
the time of my life, why the latter one saves some electricity.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Mikhail Fursov [mailto:[EMAIL PROTECTED]
>Sent: Sunday, October 29, 2006 8:56 PM
>To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
>Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
smell -
>Thread.sleep() in ActivationGroup method)
>
>On 10/29/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>
>>
>> 1) The Logging Debate That Won't Die - we don't want to encumber our
>> "production" code with logging or even with runtime enablement checks
>> for logging i.e.
>>
>>   if (logging.isDebugEnabled())
>>
>> but it's clear that some people still want to use it for debugging.
>
>
>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON
>? Doing this we solve performance issue at least .
>
>If we did this, I assume that our build becomes a two step process,
>> first pre-process the code to create  separate "buildable source",
which
>> would go into source jars and such for debugging purposes.  Then our
>> current javac/jar process.
>>
>> I'd also like to be able to work in an IDE with the pre-proc stuff
>> invisible if possible...
>
>
>This is the main problem. Backporting of your changes from the
"buildable
>source" to the "source with preprocessor" could have more overhead then
>support of a separate branch for different Java version.

Re: [general][snapshot] Windows installer


cool!

Tim Ellison wrote:

I thought it would be fun to see what a Windows installer would look
like for Harmony.

So this morning, just for kicks, I was playing with NSIS
(http://nsis.sourceforge.net/) and produced a prototype installer very
easily (kudos to them).  I'll put the source for it here [1], and you
can download an actual Harmony installer from here [2].

Caveat Emptor:  Did I mention it is a prototype already?  Feel free to
play with it.

It's easy to change the sets of files to install, right for now I
created base runtime, Class library sources, Java development, and HDK.

[1]
http://svn.apache.org/viewvc/incubator/harmony/enhanced/tools/trunk/windows_installer/
[2]
http://people.apache.org/~tellison/Apache%20Harmony%20Installer%20(r468731).exe


Regards,
Tim

Re: [drlvm][build] Where to put mmtk.jar?

Weldon,
Glad to hear that we will have separate and lightweight vmmagic jar soon.
Thanks to you and Steve!

As for today: I use Robin's site in the patch with VM helpers support to
download mmtk.jar. Let's change mmtk location when Steve adds first file
release to vmmagic.sf.net site.

+ Could you take a look at this issue:
http://issues.apache.org/jira/browse/HARMONY-2008 ?

On 10/31/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:

All,
I received a private email from Steve Blackburn regarding mmtk.jar.  There
is a new forum for discussing vmmagic at:
http://sourceforge.net/projects/vmmagic.  It seems that vmmagic API won't
be
changed impulsively as there are several projects dependent on the current
interface.

On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>
> On 10/27/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
> > On 10/27/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> > >
> > > On 10/26/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
> > > >
> > > > Hi, Mikhail, would you please educate me why this helper inlining
> work
> > > > depends on mmtk.jar? If only the unboxed classes are used, why
can't
> > > > we write a simple replacement? That's just easy.
> > >
> > >
> > > Agreed its real easy to do.  But I for one would like Harmony to
stay
> as
> > > close to MMTk standard interfaces as possible.  And the best way to
do
> > > that
> > > is download their *.class files.  Otherwise 3 years from now the
world
> > > will
> > > have MMTk vmmagic and Harmony vmmagic that are not identical.
> >
> >
> > +1
> > This also will attract MMTk developers to our project. As Robin said
our
> > project has VM, compiler, classlib and everything else needed for J2SE
> in
> > the same place. So we are more flexible for experiments then any other
> open
> > Java platform I know.
>
> Agree. As long as we have API documentation available for helper
> developers, this is a good solution.
>
> Thanks,
> xiaofeng
>
> > --
> > Mikhail Fursov
> >
> >
>

--
Weldon Washburn
Intel Enterprise Solutions Software Division

--
Mikhail Fursov

Re: [drlvm][build] Where to put mmtk.jar?


All,
I received a private email from Steve Blackburn regarding mmtk.jar.  There
is a new forum for discussing vmmagic at:
http://sourceforge.net/projects/vmmagic.  It seems that vmmagic API won't be
changed impulsively as there are several projects dependent on the current
interface.


On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:


On 10/27/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:
> On 10/27/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> >
> > On 10/26/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
> > >
> > > Hi, Mikhail, would you please educate me why this helper inlining
work
> > > depends on mmtk.jar? If only the unboxed classes are used, why can't
> > > we write a simple replacement? That's just easy.
> >
> >
> > Agreed its real easy to do.  But I for one would like Harmony to stay
as
> > close to MMTk standard interfaces as possible.  And the best way to do
> > that
> > is download their *.class files.  Otherwise 3 years from now the world
> > will
> > have MMTk vmmagic and Harmony vmmagic that are not identical.
>
>
> +1
> This also will attract MMTk developers to our project. As Robin said our
> project has VM, compiler, classlib and everything else needed for J2SE
in
> the same place. So we are more flexible for experiments then any other
open
> Java platform I know.

Agree. As long as we have API documentation available for helper
developers, this is a good solution.

Thanks,
xiaofeng

> --
> Mikhail Fursov
>
>





--
Weldon Washburn
Intel Enterprise Solutions Software Division

Re: [drlvm][sablevm] Desing of Class Unloading Support


On 10/30/06, Pavel Pervov <[EMAIL PROTECTED]> wrote:


Ignatenko vs Gagnon proposal checklist follows. :)


> In accordance with Section 2.17.8 of the JVM spec, class unloading (and
> its related native resource cleanup) can only happen when the class
> loader instance becomes unreachable.  For this to happen, we put in
> place the following things:

1- Each class loader is represented by some VM internal structure.
> [We'll call it the "class loader structure"].


This is true.



> 2- Each class loader internal structure, except (optionally) the
> bootstrap class loader, maintains a weak reference to an object
> instance of class ClassLoader (or some subclass).  The Java instance
> has some opaque pointer back to the internal VM structure.   The Java
> instance is usually created before the internal VM structure.  The
> instance constructor is usually in charge of creating the internal VM
> structure.  [We'll call it the "class loader instance"]


This is true.



> 3- Each class loader instance maintains a collection of loaded classes.
> A class/interface is never removed from this collection.  This
> collection maintains "hard" (i.e. "not weak") references to
> classes/interfaces.


This is true.



> 4- [Informative] A class loader instance is also most likely to maintain
> a collection of classes for which it has "initiated" class loading.
> This collection should use hard references (as weak references won't
> lead to earlier class loading).


This is not true. Look for the thread "[drlvm] Non-bug difference
HARMONY-1688?", where Eugene Ostrovsky desribed initiating loaders in
details with links to specification.


> 5- Each class loader instance maintains a hard reference to its parent
> class loader.  This reference is (optionally) null if the parent is the
> bootstrap class loader.


This is true. This is actually a part of delegation framework.



> 6- Each j.l.Class instance maintains a hard reference to the class
> loader instance of the class loader that has loaded it.  [This is not
> the "initiating" loaders, but really the "loading" loader].


This is true. AFAIU, this class loader is called "defining" loader for a
class.



> 7- Each class loader structure maintains a set of boolean flags, one
> flag per "non-nursery" garbage collected area (even when thread-local
> heaps are used).  The flag is set when an instance of a class loaded by
> this class leader is moved into the related GC-area.  The flag is unset
> when the GC-area is emptied, or (optionally) when it can be determined
> that no instance of a class loaded by this class loader remains in the
> GC-area.  This is best implemented as follows: a) use an unconditional
> write of "true" in the flag every time an object is moved into the
> GC-area by the garbage collector, b) unset the related flag in "all"
> class loader structures just before collecting a GC-area, then setting
> the flag back when an object survives in the area.


Requires identification of object' class type during GC. Will most
probably degrade GC performance.



Good point.  To get an idea of how much impact on performance, it would have
to be measured.


8- Each method invocation frame maintains a hard reference to either its
> surrounding instance (in case of instance methods, i.e. (invokevirtual,
> invokeinterface, and invokespecial) or its surrounding class
> (invokestatic).  This is already required for synchronized methods
> (it's not a good idea to allow the instance to be collected before the
> end of a synchronized instance method call; yep, learned the hard way
> in SableVM...)  So, the "overhead" is quite minimal.  The importance of
> this is in the correctness of not letting a class loader to die while a
> static/instance method of a class loaded by it is still active, leading
> to premature release of native resources (such as jitted code, etc.).


Not generally true for optimizing JITs. "This" (or "class") can be omitted
from enumeration if it is not used anywhere in the code. Generally, this
technique reduces number of registers used in the code ("register
pressure"
they call it :)).



Good point.  If a JIT inlines a method that makes zero reference to "this",
there may not be a way of identifying the class involved.


9- A little magic is required to prevent premature collection of a class
> loader instance and its loaded j.l.Class instances (see [3-] above), as
> object instances do not maintain a hard reference to their j.l.Class
> instance, yet we want to preserve the correctness of Object.getClass().
>
> So, the simplest approach is to maintain a hard reference in a class
> loader structure to its class loader instance (in addition to the weak
> reference in [2-] above).  This reference is kept always set (thus
> preventing collection of the class loader instance), except when *all*
> the following conditions are met:
> a) All nurseries are empty.
> b) All GC-area flags are unset.


This requires more involvment of a GC in unloading process a

RE: [classlib][luni] signalis interruptus in hysock

Geir, All,

I have examined class library code. It seems that the solution we
invented (return EINTR, then loop) was always in place. :-)

Few comments on understanding:
1. EINTR (=4) is renamed to HYPORT_ERROR_SOCKET_INTERRUPTED (=-9).
2. The loop is coded by means of "goto select".
3. The same pattern is dupdupduplicated several times.

I have not examined all places, though there could be paths which do not
fit the pattern. Honestly, I have examined the only path:

pollSelectRead() ->
hysock_select_read() ->
hysock_select()

Summary:
We can keep this issue open or close it as won't fix. Meanwhile we
should look for the real problem.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]
>Sent: Thursday, October 26, 2006 6:21 PM
>To: harmony-dev@incubator.apache.org
>Subject: Re: [classlib][luni] signalis interruptus in hysock
>
>
>
>Fedotov, Alexei A wrote:
>> Geir,
>>
>> Do I understand correctly that you suggest the following?
>>
>> 1. hysock_select as its name says should mimic a behavior of select,
i.
>> e. return the error code from select without changing it. It's ok to
>> print a rare debug message.
>
>Yes, that's what I had the other do (and no, I see no reason to print a
>debug message, as upper layers can print if they find an EINTR)
>
>>
>> 2. The correct place for the loop is the module where hysock_select
is
>> called, or, let me be precise, class lib guys are to fix our
networking
>> code.
>
>My plan is to fix it as fixed the other one.  It turns out that there
>are several layers between java and the OS...
>
>geir
>
>>
>>
>> With best regards,
>> Alexei Fedotov,
>> Intel Java & XML Engineering
>>
>>> -Original Message-
>>> From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED]
>>> Sent: Wednesday, October 25, 2006 10:01 AM
>>> To: harmony-dev@incubator.apache.org
>>> Subject: Re: [classlib][luni] signalis interruptus in hysock
>>>
>>>
>>>
>>> Weldon Washburn wrote:
 It seems JIRA is down for maintenance.  If HARMONY-1904 is still
open
 perhaps it makes sense to put a counter in the while (...) {
>> select...}
 loop. And after every N loops, print a warning/diagnostic message.
>>> For whom and to what end?  Why not just return EINTR (in hysock
speak)?
>>>
 The
 value for N would have to be tuned.  I don't know what the best
>> number
 would
 be. Given that 1904 patch is not the final solution, at least a
>>> diagnostic
 that hints at where the system hangs would be useful.  It might
make
>>> sense
 to even print a stack trace.   Also, I agree with Ivan below.
>> Signals
>>> bugs
 are very hard to debug.  And diagnostics can help us all understand
>> the
 corner cases better.
>>> But so far, no one has shown that the system hangs, or can hang,
simply
>>> because we return EINTR
>>>
>>> geir
>>>
 On 10/20/06, Ivan Volosyuk <[EMAIL PROTECTED]> wrote:
> On 10/20/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>
>> Ivan Volosyuk wrote:
>>> Well, I think that the solution is what Geir suggests. One think
> which
>>> bothers me is following. EINTR can happen in different places
>> and
>>> the
>>> situations can be quite rare in some circumstances. It can lead
>> to
>>> hard to reproduce stability bugs (race conditions).
>> Can you give an example?
> Half a year ago, I was working on the problem. Socket operations
get
> sometimes interrupted. We have found out that it occurs sometime
>> after
> GC. It was not quite easy as the application was quite big and
> situation - quite rare.
>
> Given the fact, that current implementation of monitor reservation
> code can stop other thread in quite random fashion we should have
>> rock
> solid support of EINTR handling everywhere the select(), poll()
>> calls
> is used.
>
> --
> Ivan
> Intel Enterprise Solutions Software Division
>
>>> We should find a
>>> way how to test the implementation.
>> +1!
>>
>> :)
>>
>> geir
>
>> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail:
[EMAIL PROTECTED]
> For additional commands, e-mail:
>> [EMAIL PROTECTED]
>

>>

Re: [general] creation of "jdktools"

Looks great Ilya, go for it.

Regards,
Tim

Ilya Neverov wrote:
> Hello,
> 
> I want to gather opinions about structure of the "jdktools" component.
> 
> I'm going to create scripts for moving tools' sources from classlib/
> to top-level directory jdktools/ and to prepare patches for build
> system for building tools from new place.
> 
> I think the following structure will be appropriate for future
> evolution of the jdktools:
> 
> jdktools/trunk/
>   build.xml
>   make/
>   doc/
>   modules/
>   jre/ #  keytool, tool launcher go here
>  build.xml #  classes go to jdk/jre/lib/tools.jar
>  make/
>  src/
>   jdk/ #  javac, jarsigner go here
>  build.xml #  classes go to jdk/lib/tools.jar
>  make/
>  src/
>   jdwp/#  separate module for large component
>  build.xml
>  make/
>  src/
> 
> Assumptions which look reasonable for jdktool's build subsystem:
> 
> 1) it works in presence of built classlib (as HDK binaries or as a
> result of classlib phase of overall build);
> 2) the 'jre' module is always built before building 'jdk' to provide
> generic tool launcher and the jre/lib/tools.jar. Probably it will be
> easy to obtain these items from HDK.
> 
> I'm rather newbie in the Harmony build system so your thoughts will be
> very helpful.
> 
> Thank you
> -Ilya
> 
> 
> On 10/19/06, Ilya Neverov <[EMAIL PROTECTED]> wrote:
>> Hi Geir,
>>
>> Looks like that creating the "jdktools" source tree and build was
>> shaded by other tasks. I can help with preparing and checking updates
>> in the build system. Please let me know what needs to do in this area
>> (besides svn commits) to complete the task.
>>
>> I'm especially interested in completing the move to "jdktools"
>> structure since there will be a home for the JDWP code, which has beed
>> voted but still resides in JIRA. Working with SVN will be easier.
>>
>> Thanks.
>> -Ilya
>>
>> On 10/4/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>> > yep, that's the plan.   And once we have that, we can simplify the
>> > launcher as well...
>> >
>> > Tim Ellison wrote:
>> > > +1 for creating a jdktools directory.  The dependency on the classlib
>> > > launcher should be relatively light if we go with a simple tools
>> > > launcher that rewrites the tool invocation into a generic launcher
>> > > invocation.  You may recall the idea was discussed a while ago.
>> > >
>> > > So, for example,
>> > >   jdk/bin/javac -source 1.5 -J-Xmx200M  FooBar
>> > > is rewritten to
>> > >   jdk/jre/bin/java -cp jdk/lib/tools.jar;jdk/lib/ecj.jar -Xmx200M
>> > > org.apache.harmony.tools.javac.Main -source 1.5 FooBar
>> > >
>> > > and so on.
>> > >
>> > > Regards,
>> > > Tim
>> > >
>> > > Geir Magnusson Jr. wrote:
>> > >> Geir Magnusson Jr. wrote:
>> > >>> Now that we have javac, javah, javap (if Tim votes ;) and
>> keytool, I'd
>> > >>> like to organize these and add them to the next snapshot.
>> > >> My bad - the javap isn't being voted on yet.  I was thinking of
>> the jdwp
>> > >> vote... sorry
>> > >>
>> > >>> So I propose adding a new top-level directory called "jdktools"
>> (and
>> > >>> rename "tools" to "project_tools") and create a build target that -
>> > >>> with a  dependency on classlib for the launcher - creates the
>> 'stuff'
>> > >>> needed to fill into the JDK.
>> > >>>
>> > >>> Any comments?
>> > >>>
>> > >>> geir
>> > >>>
>> > -
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: [EMAIL PROTECTED]
>> > For additional commands, e-mail: [EMAIL PROTECTED]
>> >
>> >
>>
>> -- 
>> Thank you.
>> Ilya Neverov,
>> Intel Middleware Products Division
>>
> 

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [general] creation of "jdktools"

2006-10-30 Thread Ilya Neverov

Hello,

I want to gather opinions about structure of the "jdktools" component.

I'm going to create scripts for moving tools' sources from classlib/
to top-level directory jdktools/ and to prepare patches for build
system for building tools from new place.

I think the following structure will be appropriate for future
evolution of the jdktools:

jdktools/trunk/
  build.xml
  make/
  doc/
  modules/
  jre/ #  keytool, tool launcher go here
 build.xml #  classes go to jdk/jre/lib/tools.jar
 make/
 src/
  jdk/ #  javac, jarsigner go here
 build.xml #  classes go to jdk/lib/tools.jar
 make/
 src/
  jdwp/#  separate module for large component
 build.xml
 make/
 src/

Assumptions which look reasonable for jdktool's build subsystem:

1) it works in presence of built classlib (as HDK binaries or as a
result of classlib phase of overall build);
2) the 'jre' module is always built before building 'jdk' to provide
generic tool launcher and the jre/lib/tools.jar. Probably it will be
easy to obtain these items from HDK.

I'm rather newbie in the Harmony build system so your thoughts will be
very helpful.

Thank you
-Ilya

On 10/19/06, Ilya Neverov <[EMAIL PROTECTED]> wrote:

Hi Geir,

Looks like that creating the "jdktools" source tree and build was
shaded by other tasks. I can help with preparing and checking updates
in the build system. Please let me know what needs to do in this area
(besides svn commits) to complete the task.

I'm especially interested in completing the move to "jdktools"
structure since there will be a home for the JDWP code, which has beed
voted but still resides in JIRA. Working with SVN will be easier.

Thanks.
-Ilya

On 10/4/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
> yep, that's the plan.   And once we have that, we can simplify the
> launcher as well...
>
> Tim Ellison wrote:
> > +1 for creating a jdktools directory.  The dependency on the classlib
> > launcher should be relatively light if we go with a simple tools
> > launcher that rewrites the tool invocation into a generic launcher
> > invocation.  You may recall the idea was discussed a while ago.
> >
> > So, for example,
> >   jdk/bin/javac -source 1.5 -J-Xmx200M  FooBar
> > is rewritten to
> >   jdk/jre/bin/java -cp jdk/lib/tools.jar;jdk/lib/ecj.jar -Xmx200M
> > org.apache.harmony.tools.javac.Main -source 1.5 FooBar
> >
> > and so on.
> >
> > Regards,
> > Tim
> >
> > Geir Magnusson Jr. wrote:
> >> Geir Magnusson Jr. wrote:
> >>> Now that we have javac, javah, javap (if Tim votes ;) and keytool, I'd
> >>> like to organize these and add them to the next snapshot.
> >> My bad - the javap isn't being voted on yet.  I was thinking of the jdwp
> >> vote... sorry
> >>
> >>> So I propose adding a new top-level directory called "jdktools" (and
> >>> rename "tools" to "project_tools") and create a build target that -
> >>> with a  dependency on classlib for the launcher - creates the 'stuff'
> >>> needed to fill into the JDK.
> >>>
> >>> Any comments?
> >>>
> >>> geir
> >>>
> -
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

--
Thank you.
Ilya Neverov,
Intel Middleware Products Division

Re: [general][snapshot] Windows installer

On 10/30/06, Tim Ellison <[EMAIL PROTECTED]> wrote:

Mikhail Fursov wrote:
> I support your choice. NSIS 2 is the best freely available installer I
know
> (and worked with).

Glad to hear it, though it was purely serendipitous -- I've never used
it before.  Since you have some knowledge in this area please go in and
fix up my hacks ;-)

Ok, I'll take a look.

The main benefit of NSIS
for me is an ability to extend it easily. If you miss a feature - just
implement it in C++.
Or look if someone else already did it for you :)

--
Mikhail Fursov

Re: [drlvm][sablevm] Desing of Class Unloading Support

2006-10-30 Thread Pavel Pervov


Ignatenko vs Gagnon proposal checklist follows. :)



In accordance with Section 2.17.8 of the JVM spec, class unloading (and
its related native resource cleanup) can only happen when the class
loader instance becomes unreachable.  For this to happen, we put in
place the following things:


1- Each class loader is represented by some VM internal structure.

[We'll call it the "class loader structure"].



This is true.




2- Each class loader internal structure, except (optionally) the
bootstrap class loader, maintains a weak reference to an object
instance of class ClassLoader (or some subclass).  The Java instance
has some opaque pointer back to the internal VM structure.   The Java
instance is usually created before the internal VM structure.  The
instance constructor is usually in charge of creating the internal VM
structure.  [We'll call it the "class loader instance"]



This is true.




3- Each class loader instance maintains a collection of loaded classes.
A class/interface is never removed from this collection.  This
collection maintains "hard" (i.e. "not weak") references to
classes/interfaces.



This is true.




4- [Informative] A class loader instance is also most likely to maintain
a collection of classes for which it has "initiated" class loading.
This collection should use hard references (as weak references won't
lead to earlier class loading).



This is not true. Look for the thread "[drlvm] Non-bug difference
HARMONY-1688?", where Eugene Ostrovsky desribed initiating loaders in
details with links to specification.



5- Each class loader instance maintains a hard reference to its parent
class loader.  This reference is (optionally) null if the parent is the
bootstrap class loader.



This is true. This is actually a part of delegation framework.




6- Each j.l.Class instance maintains a hard reference to the class
loader instance of the class loader that has loaded it.  [This is not
the "initiating" loaders, but really the "loading" loader].



This is true. AFAIU, this class loader is called "defining" loader for a
class.




7- Each class loader structure maintains a set of boolean flags, one
flag per "non-nursery" garbage collected area (even when thread-local
heaps are used).  The flag is set when an instance of a class loaded by
this class leader is moved into the related GC-area.  The flag is unset
when the GC-area is emptied, or (optionally) when it can be determined
that no instance of a class loaded by this class loader remains in the
GC-area.  This is best implemented as follows: a) use an unconditional
write of "true" in the flag every time an object is moved into the
GC-area by the garbage collector, b) unset the related flag in "all"
class loader structures just before collecting a GC-area, then setting
the flag back when an object survives in the area.



Requires identification of object' class type during GC. Will most
probably degrade GC performance.




8- Each method invocation frame maintains a hard reference to either its
surrounding instance (in case of instance methods, i.e. (invokevirtual,
invokeinterface, and invokespecial) or its surrounding class
(invokestatic).  This is already required for synchronized methods
(it's not a good idea to allow the instance to be collected before the
end of a synchronized instance method call; yep, learned the hard way
in SableVM...)  So, the "overhead" is quite minimal.  The importance of
this is in the correctness of not letting a class loader to die while a
static/instance method of a class loaded by it is still active, leading
to premature release of native resources (such as jitted code, etc.).



Not generally true for optimizing JITs. "This" (or "class") can be omitted
from enumeration if it is not used anywhere in the code. Generally, this
technique reduces number of registers used in the code ("register pressure"
they call it :)).




9- A little magic is required to prevent premature collection of a class
loader instance and its loaded j.l.Class instances (see [3-] above), as
object instances do not maintain a hard reference to their j.l.Class
instance, yet we want to preserve the correctness of Object.getClass().

So, the simplest approach is to maintain a hard reference in a class
loader structure to its class loader instance (in addition to the weak
reference in [2-] above).  This reference is kept always set (thus
preventing collection of the class loader instance), except when *all*
the following conditions are met:
a) All nurseries are empty.
b) All GC-area flags are unset.



This requires more involvment of a GC in unloading process and affects GC
code more. In DRLVM, GC is designed to be a replaceable component. Moreover,
we already have 3 different working GCs and MMTk on the way. So, including
GC into the design is not a good idea for DRLVM.




In addition,I highly recommend using the approach proposed in Chapter 3

of http://sablevm.org/people/egagnon/gagnon-phd.pdf for managing
class-loader related m

Re: [drlvm][em] Value-profiling implementation is avaible.


Hi Yuri.
Value profile is a profile that is really needed for set of JIT
optimizations.
What do you want to improve in JIT exactly?

On 10/30/06, Yuri Kashnikoff <[EMAIL PROTECTED]> wrote:


Hi!
I'am working on value-profiling implementation. Implementation in EM
is alredy avaible. Now I'am trying to implement and use it in JIT. I
can answer any questions about current implementation. I have used the
straigth algorithm (cacthing FIRST N values) as default and as
advanced algorithm the Top-N-Value algorithm from the B. Calder, P.
Feller "Value Profiling and Optimisation". I'll appreciate any
questions. Please review and comment.





--
Mikhail Fursov

Re: [general][snapshot] Windows installer

Mikhail Fursov wrote:
> I support your choice. NSIS 2 is the best freely available installer I know
> (and worked with).

Glad to hear it, though it was purely serendipitous -- I've never used
it before.  Since you have some knowledge in this area please go in and
fix up my hacks ;-)

Regards,
Tim

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [jira] Updated: (HARMONY-2012) [drlvm][em] value-profling implemenation


Sorry, this is my first contribution.

[drlvm][em] Value-profiling implementation is avaible.


Hi!
I'am working on value-profiling implementation. Implementation in EM
is alredy avaible. Now I'am trying to implement and use it in JIT. I
can answer any questions about current implementation. I have used the
straigth algorithm (cacthing FIRST N values) as default and as
advanced algorithm the Top-N-Value algorithm from the B. Calder, P.
Feller "Value Profiling and Optimisation". I'll appreciate any
questions. Please review and comment.

Re: [build] Building on Eclipse - FYI

Hi Sveta,

Thanks for doing that - I have had a look through your patch and it looks
really good.

Your suggestions for changes to the DRLVM document look good too.  I
couldn't really find any good links for getting started with Java
programming in Eclipse on their website, the only thing that came close was
this - http://help.eclipse.org/help32/index.jsp, which has some getting
started tutorials within the page.  There might be some better articles out
there on other sites though, or we could just direct people to the help
system or 'cheat sheets' within their Eclipse installation.

Thanks,

Sian

On 30/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:

Sian,

Taking into consideration your comments, I've opened a new JIRA issue
[http://issues.apache.org/jira/browse/HARMONY-2009] and have created a
patch for the page "Developing Apache Harmony Class-library Code with
Eclipse". Would be great, if you find a chance to look it through.
Hope we'll continue working at developing this aspect of documentation.
:)

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED]
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian

On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > I encountered a problem today building on Eclipse, and I just
> thought
> > I'd
> > > post about it here in case anyone sees the same problem in the
> > future.  I
> > > was getting the error:
> > >
> > >
> > > BUILD FAILED
> > > *
> > >
> > > C:\eclipse32harmony\eclipse\workspace\Harmony\build.xml:108: The
> > following
> > > error occurred while executing this line:
> > >
> > >
> C:\eclipse32harmony\eclipse\workspace\Harmony\make\build-java.xml:127:
> > The
> > > Eclipse compiler class for Ant could not be found. Please place
the
> ECJ
> > JAR
> > > in ANT_HOME/lib. The JAR can copied from
> > CLASSLIB_TRUNK/depends/jars/ecj_3.2
> > > folder after the fetch-depends target has been run.
> > > *
> > >
> > > The solution to this is to copy ecj_3.2.jar into
> > > ..\plugins\org.apache.ant_1.6.5\lib and *also* change the Ant
> settings
> > to
> > > include this jar (Window > Preferences > Ant > Runtime then se

Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

2006-10-30 Thread Alexei Zakharov

Hi all,

Well, as an individual who has the tendency to pure Java programming I
will be happier if I can control things on the source-code level. I
can't say I don't like the idea about sophisticated JIT with the
powerful AI inside, but if we are talking about logging then IMHO a
good preprocessor is the thing that we need (but we may also continue
to JIT away stuff if we like). At the same time I don't feel
completely comfortable with the idea of using preprocessor to separate
J2SE sources from J2ME.

No clue about particular technology. It can be SableCC, something
custom-made, velocity or whatever.

Thanks,

2006/10/30, Fedotov, Alexei A <[EMAIL PROTECTED]>:

   Premature optimization is the root of all evil
   Donald Knuth

>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON

+1

I believe a computer should adapt to my way of thinking. I prefer a
simple and readable code to an efficient one since the former one saves
the time of my life, why the latter one saves some electricity.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Mikhail Fursov [mailto:[EMAIL PROTECTED]
>Sent: Sunday, October 29, 2006 8:56 PM
>To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
>Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
smell -
>Thread.sleep() in ActivationGroup method)
>
>On 10/29/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>
>>
>> 1) The Logging Debate That Won't Die - we don't want to encumber our
>> "production" code with logging or even with runtime enablement checks
>> for logging i.e.
>>
>>   if (logging.isDebugEnabled())
>>
>> but it's clear that some people still want to use it for debugging.
>
>
>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON
>? Doing this we solve performance issue at least .
>
>If we did this, I assume that our build becomes a two step process,
>> first pre-process the code to create  separate "buildable source",
which
>> would go into source jars and such for debugging purposes.  Then our
>> current javac/jar process.
>>
>> I'd also like to be able to work in an IDE with the pre-proc stuff
>> invisible if possible...
>
>
>This is the main problem. Backporting of your changes from the
"buildable
>source" to the "source with preprocessor" could have more overhead then
>support of a separate branch for different Java version.

--
Alexei Zakharov,
Intel Enterprise Solutions Software Division

Re: [classlib] Preprocessor

Etienne Gagnon wrote:
>> Chris Gray wrote:
>>> For JavaME I think it's the only way we'd be able to maintain a single
>>> source tree. We need to be able to "#ifdef out" references to classes
>>> we don't have, methods we don't implement, etc..
>>>
>>> That much being said, I don't have a recommendation for a tool to use.
>>> Java syntax is sufficiently C-like that cpp is probably do-able, but
>>> we'd probably stumble over a whole series of gotcha's, and cpp isn't
>>> exactly [deity]'s gift to preprocessing anyway. Maybe one of the
>>> aspect-oriented tools (with which I am not at all familiar) could be a
>>> better bet?
> 
> You could always do "clean" source-to-source processing using
> SableCC...:-)  Java is a nice language to parse, so you could do some
> clean parsing, instead of the dumb "unstructured text" replacement of
> preprocessors.
> 
> Actually, if all you need if "ifdef'ing out" undesirable references, it
> could be done by "hiding" modification directives in structured
> comments,  so that these comment remain "javac" invisible.  This way you
> could make it such that:
> 1- Plain source compilation -> j2se .
> 2- Structured processed source compilation -> j2me .

I agree, ensuring that the original source remains compilable can be a
great benefit.

Besides in-lining #ifdef's you can also maintain look-aside files with a
description of the smaller configurations.  That helps to avoid code
clutter though of course you may prefer to be marking-up the code
in-line if it is not simply removing whole types/methods.

The other thing to remember is that methods that appear in the smaller
configurations must only be implemented in terms of methods that appear
in the smaller configurations.  For example, you may have to rewrite
regular IO to not be implemented in terms of NIO so that it remains
viable in configurations that don't have NIO.  In Harmony we have a
common 'platform IO layer' used for both modules.

Where you do go through a source-to-source transform stage it helps of
you can collaborate with the second-stage compiler to pass through
original line number info (a la JSR45) otherwise debugging gets a bit
tiresome.

Regards,
Tim

> If you need it, there are 2 or 3 Java 1.5 grammars available for SableCC
> (different parsing approaches, not different syntax!).  As I said, Java
> is a pleasure to parse when compared to C & C++.
> 
> It's just an idea, of course...  [I know that people can start religious
> wars about pre-processing; that's why I am suggesting a clean approach,
> so that j2se people don't have to pre-process].
> 
> Etienne
> 

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [DRLVM][GC] parallel compaction and wasted virtual space


Perfect, thanks Mikhail

On 10/30/06, Mikhail Fursov <[EMAIL PROTECTED]> wrote:


Is it the same?
http://citeseer.ist.psu.edu/630853.html
or
http://www.cs.technion.ac.il/~erez/Papers/parallel-compaction.ps

On 10/30/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:
>
> Hi,
>   Does anyone have an accessible reference to the OOPSLA paper "An
> efficient
> parallel heap compaction algorithm" by Abuaiadh, Ossia, Petrank,
> Silbershtein that is cited as a reference in the paper Xiao Feng
> points
> to below? All my google searches lead to the ACM Portal :-)
>
> Thanks,
> Rana
>
>
> On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
> >
> > Hi, all, the plan for GCv5 parallel compaction is to apply the idea of
> > Compressor [1]. But it has an issue I want to discuss with you.
> > Compressor needs to reserve an unmapped virtual space for compaction.
> > The size of the reserved part is the same as that of copy reserve
> > space in a semi-space collector. This means about that part of the
> > virtual space is unusable for the JVM. In a typical setting, the
> > wasted part is half size of the total compaction space. If we have 1GB
> > physical memory, the JVM is ok for Compressor because the virtual
> > space is large enough to wast half; but if the phsical memory is >2GB,
> > Compressor may have a problem in 32bit machine: some of phsical mapped
> > space might be wasted.
> >
> > Any opinion on this?
> >
> > Thanks,
> > xiaofeng
> >
> > [1] http://www.cs.technion.ac.il/~erez/Papers/compressor-pldi.pdf
> >
>
>


--
Mikhail Fursov

Re: [general][snapshot] Windows installer


Tim,
I support your choice. NSIS 2 is the best freely available installer I know
(and worked with).

On 10/30/06, Tim Ellison <[EMAIL PROTECTED]> wrote:


I thought it would be fun to see what a Windows installer would look
like for Harmony.

So this morning, just for kicks, I was playing with NSIS
(http://nsis.sourceforge.net/) and produced a prototype installer very
easily (kudos to them).  I'll put the source for it here [1], and you
can download an actual Harmony installer from here [2].

Caveat Emptor:  Did I mention it is a prototype already?  Feel free to
play with it.

It's easy to change the sets of files to install, right for now I
created base runtime, Class library sources, Java development, and HDK.

[1]

http://svn.apache.org/viewvc/incubator/harmony/enhanced/tools/trunk/windows_installer/
[2]

http://people.apache.org/~tellison/Apache%20Harmony%20Installer%20(r468731).exe


Regards,
Tim

--

Tim Ellison ([EMAIL PROTECTED])





--
Mikhail Fursov

Re: [DRLVM][GC] parallel compaction and wasted virtual space

Is it the same?
http://citeseer.ist.psu.edu/630853.html
or
http://www.cs.technion.ac.il/~erez/Papers/parallel-compaction.ps

On 10/30/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote:

Hi,
  Does anyone have an accessible reference to the OOPSLA paper "An
efficient
parallel heap compaction algorithm" by Abuaiadh, Ossia, Petrank,
Silbershtein that is cited as a reference in the paper Xiao Feng
points
to below? All my google searches lead to the ACM Portal :-)

Thanks,
Rana

On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:
>
> Hi, all, the plan for GCv5 parallel compaction is to apply the idea of
> Compressor [1]. But it has an issue I want to discuss with you.
> Compressor needs to reserve an unmapped virtual space for compaction.
> The size of the reserved part is the same as that of copy reserve
> space in a semi-space collector. This means about that part of the
> virtual space is unusable for the JVM. In a typical setting, the
> wasted part is half size of the total compaction space. If we have 1GB
> physical memory, the JVM is ok for Compressor because the virtual
> space is large enough to wast half; but if the phsical memory is >2GB,
> Compressor may have a problem in 32bit machine: some of phsical mapped
> space might be wasted.
>
> Any opinion on this?
>
> Thanks,
> xiaofeng
>
> [1] http://www.cs.technion.ac.il/~erez/Papers/compressor-pldi.pdf
>

--
Mikhail Fursov

[general][snapshot] Windows installer

I thought it would be fun to see what a Windows installer would look
like for Harmony.

So this morning, just for kicks, I was playing with NSIS
(http://nsis.sourceforge.net/) and produced a prototype installer very
easily (kudos to them).  I'll put the source for it here [1], and you
can download an actual Harmony installer from here [2].

Caveat Emptor:  Did I mention it is a prototype already?  Feel free to
play with it.

It's easy to change the sets of files to install, right for now I
created base runtime, Class library sources, Java development, and HDK.

[1]
http://svn.apache.org/viewvc/incubator/harmony/enhanced/tools/trunk/windows_installer/
[2]
http://people.apache.org/~tellison/Apache%20Harmony%20Installer%20(r468731).exe


Regards,
Tim

-- 

Tim Ellison ([EMAIL PROTECTED])

Re: [DRLVM][GC] parallel compaction and wasted virtual space


Hi,
 Does anyone have an accessible reference to the OOPSLA paper "An efficient
parallel heap compaction algorithm" by Abuaiadh, Ossia, Petrank,
Silbershtein that is cited as a reference in the paper Xiao Feng points
to below? All my google searches lead to the ACM Portal :-)

Thanks,
Rana


On 10/27/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:


Hi, all, the plan for GCv5 parallel compaction is to apply the idea of
Compressor [1]. But it has an issue I want to discuss with you.
Compressor needs to reserve an unmapped virtual space for compaction.
The size of the reserved part is the same as that of copy reserve
space in a semi-space collector. This means about that part of the
virtual space is unusable for the JVM. In a typical setting, the
wasted part is half size of the total compaction space. If we have 1GB
physical memory, the JVM is ok for Compressor because the virtual
space is large enough to wast half; but if the phsical memory is >2GB,
Compressor may have a problem in 32bit machine: some of phsical mapped
space might be wasted.

Any opinion on this?

Thanks,
xiaofeng

[1] http://www.cs.technion.ac.il/~erez/Papers/compressor-pldi.pdf

RE: [build] Building on Eclipse - FYI

Geir, about your suggestion

> I think that if we have an eclipse document, and the info is accurate,
I 
> say why not?

I also think that the more docs the better :) issues that I see with
this particular doc:
- info is not always up-to-date: old cmd options, old java executable
name, many other things I might not know of
- too many dependencies on Eclipse - many screenshots that quickly
become (or have become) outdated because UI in Eclipse can change. These
are sometimes just not needed because they clutter up the page and don't
explain much.
- other pages, like the Classlib page for eclipse, edited in issue
HARMONY-2009 today, repeat some info.

I like the idea of having a nice getting started - as a logical
continuation of your Quick Helps. All I wanted to say: don't think that
effort and further maintenance are minimal here :) I might try to help
with the doc, though there seem to be many other things I'll need to do.

Thank you, 
Nadya Morozova
 

-Original Message-
From: Geir Magnusson Jr. [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 3:23 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI



Sian January wrote:
> I'm actually not sure if that first page is up to date or not  (
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclip
se.html). 
> 
> I think it should be but I have not ever been able to get it to work
> myself.  Can anyone else confirm whether it is still in date or not?
> 
> Also I'm just wondering whether we really need steps 5-14 of "Running
an
> Application in Eclipse" or any of "Debugging an Application in
Eclipse"?
> There doesn't seem to be anything Harmony-specific in those steps,
it's 
> just
> about using Eclipse.  I know Eclipse itself contains that kind of
> documentation in its help system so IMHO it seems like it could be a
little
> redundant to duplicate that on the Harmony website.
> 

I think that if we have an eclipse document, and the info is accurate, I

say why not?

We're trying to suck people into the whirling vortex that is our little 
project, and if it's someone that isn't an eclipse user, but was 
intrigued by our ability to run it, etc, etc, it seem to be harmless.  I

assume that it won't change often, so the maintenance is minor?

geir

> Thanks,
> 
> Sian
> 
> 
> On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]>
wrote:
>>
>> Sian,
>>
>> Thanks a lot for the detailed description of what I should do. I'll
>> follow your recommendations. :)
>>
>> I'll investigate what we have already written about Eclipse to avoid
>> redundant info and repeating pages.
>>
>> What I've just come across are:
>> 1) The page describing how to set up Eclipse to develop Java code in
>> Apache Harmony
>>
[http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
>> pse.html];
>> 2) The sections "Running an Application in Eclipse" and "Debugging an
>> Application in Eclipse "in the Getting Started with DRL
>>
[file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
>> pse_Hello_world]
>>
>> Would be great, if you find a chance to check whether the
aforementioned
>> info is up-to-date :)
>>
>> Cheers,
>> Sveta
>> -Original Message-
>> From: Sian January [mailto:[EMAIL PROTECTED]
>> Sent: Friday, October 27, 2006 3:34 PM
>> To: harmony-dev@incubator.apache.org
>> Subject: Re: [build] Building on Eclipse - FYI
>>
>> Hi Sveta,
>>
>> That sounds like a great idea.  I think the two main things you need
to
>> do
>> extra on Eclipse on Windows are as follows:
>>
>> 1. Make a copy of vsvars32.bat from your Visual Studio install
>> directory.
>> If you have chosen the defaults when installing you will find it in
>> "C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
>> Copy it
>> to somewhere convenient such as the desktop and add the following
line
>> (just
>> below the last line that begins "@set..."):
>> start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
>> (where ... is the path to your eclipse installation).  NB. using
>> "-vmargs
>> -Xmx512M" is optional, but helpful to stop Eclipse running out of
>> memory.
>> Now just double click on this file to start Eclipse.
>>
>> 2. After you have started Eclipse and checked out Harmony from SVN,
copy
>> ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and
change
>> the
>> Ant settings to include this jar (Window > Preferences > Ant >
Runtime
>> then
>> select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
>> from
>> the org.apache.ant_1.6.5\lib directory).
>>
>> If you're happy to add that to the document that would be great.
>>
>> On Linux you will also need to do 2, but I'm not sure if there's an
>> equivalent to 1 or not.  Has anyone else tried building in Eclipse on
>> Linux?
>>
>> Thanks,
>>
>> Sian
>>
>>
>> On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]>
wrote:
>> >
>> > Folks,
>> >
>> > I see that we do need one more building doc describing Eclipse
>> > specifics.
>> > The ma

RE: [build] Building on Eclipse - FYI

+1

Thank you, 
Nadya Morozova

-Original Message-
From: Konovalova, Svetlana [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 5:05 PM
To: harmony-dev@incubator.apache.org
Subject: RE: [build] Building on Eclipse - FYI

Well, my 2cents:

>My concern is that if there is too much documentation and it's not well
>structured then people won't be inclined to read any of it.  If 50% of
the
>"Getting Started with DRLVM" page is actually about Eclipse, my feeling
is
>that it won't be as useful for people who actually want to get started
with
>DRLVM than it would be if it was half the size and more concise.  

I'm with you here. It goes without saying that when a person comes
across a long description, he/she just skips it. In our documentation we
should provide DRLVM-related information only. My suggestions are:
1) To remove "just-using-Eclipse" information from the "Getting Started
with DRLVM" doc
[http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started
] to make the doc purely DRLVM-oriented;
2) To add certain links from the doc to external sources providing
general information about Eclipse.

How about that?

AFAIU we should get two docs guiding the Eclipse users:
1) The "Getting Started with DRLVM" doc including step-by-step
instructions on running a Java application in the Eclipse environment.
[http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started
];
2) The "Developing Apache Harmony class library code with Eclipse" page
providing instructions to help users set up Eclipse to develop Java code
in Apache Harmony.  
[http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
pse.html]

Any comments?

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 4:07 PM
To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
Subject: Re: [build] Building on Eclipse - FYI

My concern is that if there is too much documentation and it's not well
structured then people won't be inclined to read any of it.  If 50% of
the
"Getting Started With DRLVM" page is actually about Eclipse, my feeling
is
that it won't be as useful for people who actually want to get started
with
DRLVM than it would be if it was half the size and more concise.  In
general
I think having Eclipse-related documentation on its own page (or
inline where it's relevant) is a good thing, I was just commenting on
the
amount of seemingly unrelated Eclipse documentation that is currently
included in the DRLVM page (
http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started.
html).
Just my 2p worth...

Thanks,

Sian

On 30/10/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>
>
>
> Sian January wrote:
> > I'm actually not sure if that first page is up to date or not  (
> >
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclip
se.html
> ).
> >
> > I think it should be but I have not ever been able to get it to work
> > myself.  Can anyone else confirm whether it is still in date or not?
> >
> > Also I'm just wondering whether we really need steps 5-14 of
"Running an
> > Application in Eclipse" or any of "Debugging an Application in
Eclipse"?
> > There doesn't seem to be anything Harmony-specific in those steps,
it's
> > just
> > about using Eclipse.  I know Eclipse itself contains that kind of
> > documentation in its help system so IMHO it seems like it could be a
> little
> > redundant to duplicate that on the Harmony website.
> >
>
> I think that if we have an eclipse document, and the info is accurate,
I
> say why not?
>
> We're trying to suck people into the whirling vortex that is our
little
> project, and if it's someone that isn't an eclipse user, but was
> intrigued by our ability to run it, etc, etc, it seem to be harmless.
I
> assume that it won't change often, so the maintenance is minor?
>
> geir
>
> > Thanks,
> >
> > Sian
> >
> >
> > On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]>
wrote:
> >>
> >> Sian,
> >>
> >> Thanks a lot for the detailed description of what I should do. I'll
> >> follow your recommendations. :)
> >>
> >> I'll investigate what we have already written about Eclipse to
avoid
> >> redundant info and repeating pages.
> >>
> >> What I've just come across are:
> >> 1) The page describing how to set up Eclipse to develop Java code
in
> >> Apache Harmony
> >> [
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
> >> pse.html];
> >> 2) The sections "Running an Application in Eclipse" and "Debugging
an
> >> Application in Eclipse "in the Getting Started with DRL
> >> [
>
file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
> >> pse_Hello_world]
> >>
> >> Would be great, if you find a chance to check whether the
> aforementioned
> >> info is up-to-date :)
> >>
> >> Cheers,
> >> Sveta
> >> -Original Message-
> >> From: Sian January [mailto:[EMAIL PROTECTED]
> >> Sent: Friday, October 27, 2006 3:34 PM
> >> To: harmony-de

RE: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Premature optimization is the root of all evil
Donald Knuth


>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON

+1

I believe a computer should adapt to my way of thinking. I prefer a
simple and readable code to an efficient one since the former one saves
the time of my life, why the latter one saves some electricity.

With best regards,
Alexei Fedotov,
Intel Java & XML Engineering

>-Original Message-
>From: Mikhail Fursov [mailto:[EMAIL PROTECTED]
>Sent: Sunday, October 29, 2006 8:56 PM
>To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
>Subject: Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code
smell -
>Thread.sleep() in ActivationGroup method)
>
>On 10/29/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>>
>>
>> 1) The Logging Debate That Won't Die - we don't want to encumber our
>> "production" code with logging or even with runtime enablement checks
>> for logging i.e.
>>
>>   if (logging.isDebugEnabled())
>>
>> but it's clear that some people still want to use it for debugging.
>
>
>Just a small idea: Let teach JIT to purge this code unless special
option
>is ON
>? Doing this we solve performance issue at least .
>
>If we did this, I assume that our build becomes a two step process,
>> first pre-process the code to create  separate "buildable source",
which
>> would go into source jars and such for debugging purposes.  Then our
>> current javac/jar process.
>>
>> I'd also like to be able to work in an IDE with the pre-proc stuff
>> invisible if possible...
>
>
>This is the main problem. Backporting of your changes from the
"buildable
>source" to the "source with preprocessor" could have more overhead then
>support of a separate branch for different Java version.
>
>
>
>
>--
>Mikhail Fursov

RE: [build] Building on Eclipse - FYI

Sveta,
I've taken a brief look at the patch, and I like most of your
corrections. The page looks neater this way, and holds important data.

One question: can't we say that some of the tips given on the page apply
to harmony code in general, not just classlib? So a side idea would be
to have one page: how to work with harmony code in Eclipse. What do you
say? 

Thank you, 
Nadya Morozova
 

-Original Message-
From: Konovalova, Svetlana [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 6:23 PM
To: harmony-dev@incubator.apache.org
Subject: RE: [build] Building on Eclipse - FYI

Sian,

Taking into consideration your comments, I've opened a new JIRA issue
[http://issues.apache.org/jira/browse/HARMONY-2009] and have created a
patch for the page "Developing Apache Harmony Class-library Code with
Eclipse". Would be great, if you find a chance to look it through. 
Hope we'll continue working at developing this aspect of documentation.
:)

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED] 
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian


On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > I encountered a problem today building on Eclipse, and I just
> thought
> > I'd
> > > post about it here in case anyone sees the same problem in the
> > future.  I
> > > was getting the error:
> > >
> > >
> > > BUILD FAILED
> > > *
> > >
> > > C:\eclipse32harmony\eclipse\workspace\Harmony\build.xml:108: The
> > following
> > > error occurred while executing this line:
> > >
> > >
> C:\eclipse32harmony\eclipse\workspace\Harmony\make\build-java.xml:127:
> > The
> > > Eclipse compiler class for Ant could not be found. Please place
the
> ECJ
> > JAR
> > > in ANT_HOME/lib. The JAR can copied from
> > CLASSLIB_TRUNK/depends/jars/ecj_3.2
> > > folder after the fetch-depends target has been run.
> > > *
> > >
> > > The solution to this is to copy ecj_3.2.jar into
> > > ..\plugins\org.apache.ant_1.6.5\lib and *also* change the Ant
> settings
> > to
> > > include this jar (Window > Preferences > Ant > Runtime then select
> > 'Global
> > > Entries' then 'Add External Jars' and add ecj_

Re: [drlvm] what's the difference between the structure VM_thread and HyThread?


Just my $0.02:
Actually VM_thread contains thread-local data. The VM_thread is a thread
local data by itself. HyThread keeps a pointer to it, so VM_thread is always
accessed from HyThread.
Some of the components, e.g. GCv4 and GCv5, keeps all thread local data in
VM_thread, but others JIT, GC_CC use more up to date method and request TLS
slots directly from TM.

Today we are in a transition from the old model when all TLS data was stored
in VM_thread to the new model - when every component requests TLS slots
independently from TM

On 10/30/06, Nikolay Kuznetsov <[EMAIL PROTECTED]> wrote:


Yes, this data is non threading and separate fields of VM_thread
structure should be replaced with independent values (data structures)
stored in TLS under separate keys.

Nik.



--
Mikhail Fursov

Re: [drlvm][sablevm] Desing of Class Unloading Support


I like it.  I don't fully understand the fine details yet.  But overall it
seems to be a clean design.  Maybe it makes sense for someone to prototype
this in drlvm.

On 10/30/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:


Hi all,

Here's a more structured proposal for a simple and effective
implementation of class unloading support.

In accordance with Section 2.17.8 of the JVM spec, class unloading (and
its related native resource cleanup) can only happen when the class
loader instance becomes unreachable.  For this to happen, we put in
place the following things:

1- Each class loader is represented by some VM internal structure.
[We'll call it the "class loader structure"].

2- Each class loader internal structure, except (optionally) the
bootstrap class loader, maintains a weak reference to an object
instance of class ClassLoader (or some subclass).  The Java instance
has some opaque pointer back to the internal VM structure.   The Java
instance is usually created before the internal VM structure.  The
instance constructor is usually in charge of creating the internal VM
structure.  [We'll call it the "class loader instance"]

3- Each class loader instance maintains a collection of loaded classes.
A class/interface is never removed from this collection.  This
collection maintains "hard" (i.e. "not weak") references to
classes/interfaces.

4- [Informative] A class loader instance is also most likely to maintain
a collection of classes for which it has "initiated" class loading.
This collection should use hard references (as weak references won't
lead to earlier class loading).

5- Each class loader instance maintains a hard reference to its parent
class loader.  This reference is (optionally) null if the parent is the
bootstrap class loader.

6- Each j.l.Class instance maintains a hard reference to the class
loader instance of the class loader that has loaded it.  [This is not
the "initiating" loaders, but really the "loading" loader].

7- Each class loader structure maintains a set of boolean flags, one
flag per "non-nursery" garbage collected area (even when thread-local
heaps are used).  The flag is set when an instance of a class loaded by
this class leader is moved into the related GC-area.  The flag is unset
when the GC-area is emptied, or (optionally) when it can be determined
that no instance of a class loaded by this class loader remains in the
GC-area.  This is best implemented as follows: a) use an unconditional
write of "true" in the flag every time an object is moved into the
GC-area by the garbage collector, b) unset the related flag in "all"
class loader structures just before collecting a GC-area, then setting
the flag back when an object survives in the area.

8- Each method invocation frame maintains a hard reference to either its
surrounding instance (in case of instance methods, i.e. (invokevirtual,
invokeinterface, and invokespecial) or its surrounding class
(invokestatic).  This is already required for synchronized methods
(it's not a good idea to allow the instance to be collected before the
end of a synchronized instance method call; yep, learned the hard way
in SableVM...)  So, the "overhead" is quite minimal.  The importance of
this is in the correctness of not letting a class loader to die while a
static/instance method of a class loaded by it is still active, leading
to premature release of native resources (such as jitted code, etc.).

9- A little magic is required to prevent premature collection of a class
loader instance and its loaded j.l.Class instances (see [3-] above), as
object instances do not maintain a hard reference to their j.l.Class
instance, yet we want to preserve the correctness of Object.getClass().

So, the simplest approach is to maintain a hard reference in a class
loader structure to its class loader instance (in addition to the weak
reference in [2-] above).  This reference is kept always set (thus
preventing collection of the class loader instance), except when *all*
the following conditions are met:
a) All nurseries are empty.
b) All GC-area flags are unset.

Actually, for making this practical and preserving correctness, it's a
little trickier.  It requires a 2-step process, much like the
object-finalization dance.  Here's a typical example:

On a major collection, where all nurseries are collected, and some (but
not necessary all) other GC-areas are collected, we do the following
sequence of actions:
a) All class loader structures are visited.  All flags related to
  non-nursery GC-areas that we intend to collect are unset.  If this
  leads to *all* flags to be unset, the hard reference to the class
  loader instance is set to NULL (thus enabling, possibly, the
  collection of the class loader instance).

b) The garbage collection cycle is started and proceeds as usual.
  Note that the work mandated in [7-] above is also done, which might
  lead to setting back some flags in class loader structures that had
  all their flags unset in [a)].

c) Aft

RE: [build] Building on Eclipse - FYI

2006-10-30 Thread Konovalova, Svetlana

Sian,

Taking into consideration your comments, I've opened a new JIRA issue
[http://issues.apache.org/jira/browse/HARMONY-2009] and have created a
patch for the page "Developing Apache Harmony Class-library Code with
Eclipse". Would be great, if you find a chance to look it through. 
Hope we'll continue working at developing this aspect of documentation.
:)

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED] 
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian


On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > I encountered a problem today building on Eclipse, and I just
> thought
> > I'd
> > > post about it here in case anyone sees the same problem in the
> > future.  I
> > > was getting the error:
> > >
> > >
> > > BUILD FAILED
> > > *
> > >
> > > C:\eclipse32harmony\eclipse\workspace\Harmony\build.xml:108: The
> > following
> > > error occurred while executing this line:
> > >
> > >
> C:\eclipse32harmony\eclipse\workspace\Harmony\make\build-java.xml:127:
> > The
> > > Eclipse compiler class for Ant could not be found. Please place
the
> ECJ
> > JAR
> > > in ANT_HOME/lib. The JAR can copied from
> > CLASSLIB_TRUNK/depends/jars/ecj_3.2
> > > folder after the fetch-depends target has been run.
> > > *
> > >
> > > The solution to this is to copy ecj_3.2.jar into
> > > ..\plugins\org.apache.ant_1.6.5\lib and *also* change the Ant
> settings
> > to
> > > include this jar (Window > Preferences > Ant > Runtime then select
> > 'Global
> > > Entries' then 'Add External Jars' and add ecj_3.2.jar from the
> > > org.apache.ant_1.6.5\lib directory).
> > >
> > > I don't know if many people build on Eclipse, and I know there
have
> been
> > > some conversations about having too many instructions on the
> website,
> > but
> > > since the steps are slightly different for Eclipse I wondered if
it
> > might be
> > > worth augmenting the instructions or adding some Eclipse-specific
> ones
> > at
> > > some point?
> > >
> > > Sian
> > >
> > > --
> > > Sian January
> > >
> > > IBM Java Technology Centre, UK
> > >
> > >
> >
>
>
>
> --
> Sian January
>
> IBM Java Technology Centre, UK
>

[doc][drlvm] new docs added - JIT compiler

All, 

New documents have been added to HARMONY-2003. These are a description
of the Jitrino.OPT and .JET compilers, with a supplementary doc on the
pipeline management framework inside the JITs. The docs describe the
architecture, processes and usage of the components.

 

It would be great if somebody took a look, expressed an opinion, and, if
favorable, committed to the website.

Your review or any other feedback are most welcome,

 

Thanks, 

Nadya Morozova

 

PS: this JIRA is not the only one unresolved doc issue. Find more useful
pending patches in our database :-)

[drlvm] VM helpers inlining framework is in JIRA


All,
I've put $subj to JIRA: http://issues.apache.org/jira/browse/HARMONY-2008
Please review, comment and make it commited :)
I can answer any questions about current implementation and it's state in
this thread.

Brief list of changes:
1) gc_cc - helper written in Java added.
2) vmcore - support to bootstrap Java code for components added.
3) build - depends on mmtk.jar now (need 'build.sh update')
4) jit - new optimizations: helper inlining, inlining final fields values as
consts

Plans for this week: start to work on monitor's helpers and enable helpers
inlining optimization in all modes with Jitrino.OPT.

+
The 'linux.emconf' file in JIRA is the opt.emconf with helpers inlining
turned on. You can use this  file to run GC tests. Note, today we have only
allocation helper inlined for objects, not for arrays.

--
Mikhail Fursov

Re: [drlvm] Class unloading support

2006-10-30 Thread Etienne Gagnon

Hi Weldon,

Weldon Washburn wrote:
> I read section 3.2.3 (Class-Loader-Specific Memory) of gagnon-phd.pdf.
> Please tell me if the following is a correct interpretation.  You create
> a new memory manager that is uniquely associated with each new class
> loader.

Right.

>  All the C data structures associated with a class loader (classes,
> vtables, etc) are "malloc()ed" out of the associated memory manager.

[For those who have not read it...]

"malloc()ed" is a big word...  It is rather "simpleAlloc()ed", i.e.,
once allocated, you cannot free it (...or if you do, the "free-list"
manager is very minimal, performs no checks [you have to tell it how
much you are freeing] and no aggregation).  I do discuss this in the
Chapter, of course, and you can look at the implementation in SableVM.
[The SableVM trunk is under AL2.0 (unlike released versions)].

>  When
> the class loader becomes unreachable, then its associated memory manager is
> deallocated which automatically frees all the associated C structs
> (classes, vtables, etc.)

Yep.

Etienne

-- 
Etienne M. Gagnon, Ph.D.http://www.info2.uqam.ca/~egagnon/
SableVM:   http://www.sablevm.org/
SableCC:   http://www.sablecc.org/

signature.asc
Description: OpenPGP digital signature

[drlvm][sablevm] Desing of Class Unloading Support

2006-10-30 Thread Etienne Gagnon

Hi all,

Here's a more structured proposal for a simple and effective
implementation of class unloading support.

In accordance with Section 2.17.8 of the JVM spec, class unloading (and
its related native resource cleanup) can only happen when the class
loader instance becomes unreachable.  For this to happen, we put in
place the following things:

1- Each class loader is represented by some VM internal structure.
 [We'll call it the "class loader structure"].

2- Each class loader internal structure, except (optionally) the
 bootstrap class loader, maintains a weak reference to an object
 instance of class ClassLoader (or some subclass).  The Java instance
 has some opaque pointer back to the internal VM structure.   The Java
 instance is usually created before the internal VM structure.  The
 instance constructor is usually in charge of creating the internal VM
 structure.  [We'll call it the "class loader instance"]

3- Each class loader instance maintains a collection of loaded classes.
 A class/interface is never removed from this collection.  This
 collection maintains "hard" (i.e. "not weak") references to
 classes/interfaces.

4- [Informative] A class loader instance is also most likely to maintain
 a collection of classes for which it has "initiated" class loading.
 This collection should use hard references (as weak references won't
 lead to earlier class loading).

5- Each class loader instance maintains a hard reference to its parent
 class loader.  This reference is (optionally) null if the parent is the
 bootstrap class loader.

6- Each j.l.Class instance maintains a hard reference to the class
 loader instance of the class loader that has loaded it.  [This is not
 the "initiating" loaders, but really the "loading" loader].

7- Each class loader structure maintains a set of boolean flags, one
 flag per "non-nursery" garbage collected area (even when thread-local
 heaps are used).  The flag is set when an instance of a class loaded by
 this class leader is moved into the related GC-area.  The flag is unset
 when the GC-area is emptied, or (optionally) when it can be determined
 that no instance of a class loaded by this class loader remains in the
 GC-area.  This is best implemented as follows: a) use an unconditional
 write of "true" in the flag every time an object is moved into the
 GC-area by the garbage collector, b) unset the related flag in "all"
 class loader structures just before collecting a GC-area, then setting
 the flag back when an object survives in the area.

8- Each method invocation frame maintains a hard reference to either its
 surrounding instance (in case of instance methods, i.e. (invokevirtual,
 invokeinterface, and invokespecial) or its surrounding class
 (invokestatic).  This is already required for synchronized methods
 (it's not a good idea to allow the instance to be collected before the
 end of a synchronized instance method call; yep, learned the hard way
 in SableVM...)  So, the "overhead" is quite minimal.  The importance of
 this is in the correctness of not letting a class loader to die while a
 static/instance method of a class loaded by it is still active, leading
 to premature release of native resources (such as jitted code, etc.).

9- A little magic is required to prevent premature collection of a class
 loader instance and its loaded j.l.Class instances (see [3-] above), as
  object instances do not maintain a hard reference to their j.l.Class
 instance, yet we want to preserve the correctness of Object.getClass().

 So, the simplest approach is to maintain a hard reference in a class
 loader structure to its class loader instance (in addition to the weak
 reference in [2-] above).  This reference is kept always set (thus
 preventing collection of the class loader instance), except when *all*
 the following conditions are met:
  a) All nurseries are empty.
  b) All GC-area flags are unset.

 Actually, for making this practical and preserving correctness, it's a
 little trickier.  It requires a 2-step process, much like the
 object-finalization dance.  Here's a typical example:

 On a major collection, where all nurseries are collected, and some (but
 not necessary all) other GC-areas are collected, we do the following
 sequence of actions:
  a) All class loader structures are visited.  All flags related to
   non-nursery GC-areas that we intend to collect are unset.  If this
   leads to *all* flags to be unset, the hard reference to the class
   loader instance is set to NULL (thus enabling, possibly, the
   collection of the class loader instance).

  b) The garbage collection cycle is started and proceeds as usual.
   Note that the work mandated in [7-] above is also done, which might
   lead to setting back some flags in class loader structures that had
   all their flags unset in [a)].

  c) After the initial garbage collection is applied, and just before
   the usual treatment of weak references (where they are set to NULL
   when pointing to a co

Re: [drlvm] Class unloading support

Etienne,

I read section 3.2.3 (Class-Loader-Specific Memory) of gagnon-phd.pdf.
Please tell me if the following is a correct interpretation.  You create
a new memory manager that is uniquely associated with each new class
loader.  All the C data structures associated with a class loader (classes,
vtables, etc) are "malloc()ed" out of the associated memory manager.  When
the class loader becomes unreachable, then its associated memory manager is
deallocated which automatically frees all the associated C structs (classes,
vtables, etc.)

Everyone,
Does it make sense to try to implement Etienne's scheme?

On 10/30/06, Etienne Gagnon <[EMAIL PROTECTED]> wrote:

>>> If I get it right, in case of automagic unloading, GC does all the job
>>> without a knowledge whether it collects a class, a classloader or
>>> whatever else.
>>> Perhaps I'm missing something, but to provide a callback on class
>>> unloading, the GC must know the semantic of the object being
collected.

Am I wrong, or does this proposition imply collecting classes
independently from their class loader?  If this is the case, I have to
say that I disagree with the proposed approach.

The JVM spec says quite clearly:
2.17.8 Unloading of Classes and Interfaces

A class or interface may be unloaded if and only if its class loader is
unreachable. The bootstrap class loader is always reachable; as a
result, system classes may never be unloaded.

Just think about it.  One could take an instance "o" of a class C loaded
by L, call it (C,L), and call o.getClass().hashcode().  Store this
integer some where.  Then, "o" could die, and maybe (C,L) unloaded while
L is still reachable.  As L is still reachable, some code could do a
L.findClass("C").hashcode().  This will likely result in a different
hashcode, in full breach of the both the VM and API specifications.

In a related note, for memory management I highly encourage Drlvm to
look at Chapter 3 of http://sablevm.org/people/egagnon/gagnon-phd.pdf
that exposes a simple, yet very effective approach for managing
class-loader related memory (i.e. memory used to store internal class
data, vtables, jitted code) so that it can all be freed efficiently at
class-loader unloading time.

Etienne

--
Etienne M. Gagnon, Ph.D.http://www.info2.uqam.ca/~egagnon/
SableVM:   http://www.sablevm.org/
SableCC:   http://www.sablecc.org/

--
Weldon Washburn
Intel Enterprise Solutions Software Division

Re: [drlvm] what's the difference between the structure VM_thread and HyThread?

2006-10-30 Thread Nikolay Kuznetsov


Yes, this data is non threading and separate fields of VM_thread
structure should be replaced with independent values (data structures)
stored in TLS under separate keys.

Nik.

On 10/30/06, Fan Bin <[EMAIL PROTECTED]> wrote:

Thanks,  so you mean that there is only non threading data currently in 
VM_thread block, and maybe the VM_thread block will be replaced by TLS in 
HyThread, right?

Bin

- Original Message -
From: "Nikolay Kuznetsov" <[EMAIL PROTECTED]>
To: 
Sent: Monday, October 30, 2006 9:13 PM
Subject: Re: [drlvm] what's the difference between the structure VM_thread and 
HyThread?


> Hello,
>
> this is legacy thread structure. You're right, there was no HyThread
> at first, but VM_thread structure. Also different DRLVM modules used
> to have dedicated fields in this structure. Further it was decided to
> make threading module independent and make usage model of threading
> module the same for all other modules(i.e use thread local storage for
> module specific data). But since VM_thread structure usage was very
> heavy it was also decided to leave all non threading data in VM_thread
> and put it in TLS till better times (I mean, till this data will be
> divided into module depended parts and also put into TLS under
> different keys).
>
> Nik.
>
> On 10/30/06, Fan Bin <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> I'm now reading the lastest version of drlvm. Now I have a question about 
the thread management module. I see that there are three kinds of control blocks: 
HyThread, JVMTIThread and VM_thread. What's the difference of their rols? Especially 
HyThread and VM_thread. As far as I know, there is no HyThread at first.
>>
>> Thanks,
>> Fan Bin

Re: [drlvm] what's the difference between the structure VM_thread and HyThread?

2006-10-30 Thread Fan Bin

Thanks,  so you mean that there is only non threading data currently in 
VM_thread block, and maybe the VM_thread block will be replaced by TLS in 
HyThread, right?

Bin

- Original Message - 
From: "Nikolay Kuznetsov" <[EMAIL PROTECTED]>
To: 
Sent: Monday, October 30, 2006 9:13 PM
Subject: Re: [drlvm] what's the difference between the structure VM_thread and 
HyThread?


> Hello,
> 
> this is legacy thread structure. You're right, there was no HyThread
> at first, but VM_thread structure. Also different DRLVM modules used
> to have dedicated fields in this structure. Further it was decided to
> make threading module independent and make usage model of threading
> module the same for all other modules(i.e use thread local storage for
> module specific data). But since VM_thread structure usage was very
> heavy it was also decided to leave all non threading data in VM_thread
> and put it in TLS till better times (I mean, till this data will be
> divided into module depended parts and also put into TLS under
> different keys).
> 
> Nik.
> 
> On 10/30/06, Fan Bin <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> I'm now reading the lastest version of drlvm. Now I have a question about 
>> the thread management module. I see that there are three kinds of control 
>> blocks: HyThread, JVMTIThread and VM_thread. What's the difference of their 
>> rols? Especially HyThread and VM_thread. As far as I know, there is no 
>> HyThread at first.
>>
>> Thanks,
>> Fan Bin

Re: Re: [drlvm] Class unloading support

The point is not that it is unimportant because it is an optimization. It is
1) it seems something that is be good to have, but is not urgent
immediately 2) that given the nature of our best solution ( java tables etc.
) is risky, we may not want to experiment with it in the main branch. We
should also study other solutions.

On 10/28/06, Alex Blewitt <[EMAIL PROTECTED]> wrote:

 True, but then JIT is an optimisation that isn't mandated in the JLS
> > either :-) There are also JVMs that don't depend on a JIT, but just
> > because it isn't mandated as a standard doesn't make it any less
> > important to implement it.
> >
> > For that matter, there's nothing in the JLS that mandates how GC
> > works. It's quite possible to have a JVM that never does any GC, and
> > just sucks memory until it can't suck any more, and throw an
> > OutOfMemoryException. What the JLS does say is the order in which
> > finalise methods should be called prior to the storage being
> > reclaimed; they don't mandate that the storage must be reclaimed.
> >
> > So, just because it's not mandated doesn't mean it's not important to
> > do :-)
> >
> > Alex.
> >
>

Re: [classlib] Preprocessor (was Re: [classlib][rmi] Code smell - Thread.sleep() in ActivationGroup method)

Yes - I know AspectJ works on the bytecode and not as a pre-processor to the
source code and I don't think any other AO languages do that either.
Although I'm an advocate for AOP I think we would want to think seriously
before introducing a dependency on a non-javac compiler to Harmony.  However
it would be good for logging, and it's worth noting that AspectJ 5 can
also match based on annotations, which makes it possible to achieve quite
fine-grained logging without cluttering up the source too much with "if (
logging.isDebugEnabled())" etc.

Thanks,

Sian

On 30/10/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:

Chris Gray wrote:
> On Sunday 29 October 2006 14:04, Geir Magnusson Jr. wrote:
>> [...]
>> 3) Java ME - We've had some interest (Chris?) in looking at using the
>> Harmony classlib for ME, which can also have some differences that
might
>> be most conveniently kept in place in the main codebase.
>
> Yes, I'm still here and still waiting for an answer to my last mail
about
> bringing Mika to the incubator ...

I thought you were off trying to figure out IP provenance.

>
>> [...]
>> First, anyone think this is a good idea and second, anyone have any
>> experience with tools in this area?
>
> For JavaME I think it's the only way we'd be able to maintain a single
source
> tree. We need to be able to "#ifdef out" references to classes we don't
have,
> methods we don't implement, etc..
>
> That much being said, I don't have a recommendation for a tool to use.
Java
> syntax is sufficiently C-like that cpp is probably do-able, but we'd
probably
> stumble over a whole series of gotcha's, and cpp isn't exactly [deity]'s
gift
> to preprocessing anyway. Maybe one of the aspect-oriented tools (with
which I
> am not at all familiar) could be a better bet?

How?  Doesn't that tend to work on the bytecode?  I know that I'd be
uncomfortable with anything where there wasn't a clear source tree
produced.

geir

>
> Cheers,
>
> Chris
>

--
Sian January

IBM Java Technology Centre, UK

RE: [build] Building on Eclipse - FYI

2006-10-30 Thread Konovalova, Svetlana

Well, my 2cents:

>My concern is that if there is too much documentation and it's not well
>structured then people won't be inclined to read any of it.  If 50% of
the
>"Getting Started with DRLVM" page is actually about Eclipse, my feeling
is
>that it won't be as useful for people who actually want to get started
with
>DRLVM than it would be if it was half the size and more concise.  

I'm with you here. It goes without saying that when a person comes
across a long description, he/she just skips it. In our documentation we
should provide DRLVM-related information only. My suggestions are:
1) To remove "just-using-Eclipse" information from the "Getting Started
with DRLVM" doc
[http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started
] to make the doc purely DRLVM-oriented;
2) To add certain links from the doc to external sources providing
general information about Eclipse.

How about that?

AFAIU we should get two docs guiding the Eclipse users:
1) The "Getting Started with DRLVM" doc including step-by-step
instructions on running a Java application in the Eclipse environment.
[http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started
];
2) The "Developing Apache Harmony class library code with Eclipse" page
providing instructions to help users set up Eclipse to develop Java code
in Apache Harmony.  
[http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
pse.html]

Any comments?

Cheers,
Sveta

-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED] 
Sent: Monday, October 30, 2006 4:07 PM
To: harmony-dev@incubator.apache.org; [EMAIL PROTECTED]
Subject: Re: [build] Building on Eclipse - FYI

My concern is that if there is too much documentation and it's not well
structured then people won't be inclined to read any of it.  If 50% of
the
"Getting Started With DRLVM" page is actually about Eclipse, my feeling
is
that it won't be as useful for people who actually want to get started
with
DRLVM than it would be if it was half the size and more concise.  In
general
I think having Eclipse-related documentation on its own page (or
inline where it's relevant) is a good thing, I was just commenting on
the
amount of seemingly unrelated Eclipse documentation that is currently
included in the DRLVM page (
http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started.
html).
Just my 2p worth...

Thanks,

Sian

On 30/10/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:
>
>
>
> Sian January wrote:
> > I'm actually not sure if that first page is up to date or not  (
> >
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclip
se.html
> ).
> >
> > I think it should be but I have not ever been able to get it to work
> > myself.  Can anyone else confirm whether it is still in date or not?
> >
> > Also I'm just wondering whether we really need steps 5-14 of
"Running an
> > Application in Eclipse" or any of "Debugging an Application in
Eclipse"?
> > There doesn't seem to be anything Harmony-specific in those steps,
it's
> > just
> > about using Eclipse.  I know Eclipse itself contains that kind of
> > documentation in its help system so IMHO it seems like it could be a
> little
> > redundant to duplicate that on the Harmony website.
> >
>
> I think that if we have an eclipse document, and the info is accurate,
I
> say why not?
>
> We're trying to suck people into the whirling vortex that is our
little
> project, and if it's someone that isn't an eclipse user, but was
> intrigued by our ability to run it, etc, etc, it seem to be harmless.
I
> assume that it won't change often, so the maintenance is minor?
>
> geir
>
> > Thanks,
> >
> > Sian
> >
> >
> > On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]>
wrote:
> >>
> >> Sian,
> >>
> >> Thanks a lot for the detailed description of what I should do. I'll
> >> follow your recommendations. :)
> >>
> >> I'll investigate what we have already written about Eclipse to
avoid
> >> redundant info and repeating pages.
> >>
> >> What I've just come across are:
> >> 1) The page describing how to set up Eclipse to develop Java code
in
> >> Apache Harmony
> >> [
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
> >> pse.html];
> >> 2) The sections "Running an Application in Eclipse" and "Debugging
an
> >> Application in Eclipse "in the Getting Started with DRL
> >> [
>
file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
> >> pse_Hello_world]
> >>
> >> Would be great, if you find a chance to check whether the
> aforementioned
> >> info is up-to-date :)
> >>
> >> Cheers,
> >> Sveta
> >> -Original Message-
> >> From: Sian January [mailto:[EMAIL PROTECTED]
> >> Sent: Friday, October 27, 2006 3:34 PM
> >> To: harmony-dev@incubator.apache.org
> >> Subject: Re: [build] Building on Eclipse - FYI
> >>
> >> Hi Sveta,
> >>
> >> That sounds like a great idea.  I think the two main things you
need to
> >> do
> >> extra on Eclipse on Windows are as follows:
> >>

Re: [classlib][IBMVME]the return value of java.exe

2006-10-30 Thread Alexey Varlamov


2006/10/30, Evgueni Brevnov <[EMAIL PROTECTED]>:

Hi,

https://issues.apache.org/jira/browse/HARMONY-2006 is created with
patches :-). I created two separate patches one for DRLVM another one
for Classlib. I don't know if IBMVME requires additional changes. Even
though DestroyJavaVM will not terminate the whole process now it still
doesn't clean up resources properly. I'm going to focus on that
problem...but it seems to take much more than one day :-)

Thanks
Evgueni.

On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> Tim,
>
> I agree, we shouldn't kill entire process. Unfortunately, it is how
> current implementation does. Actually, shutdown process is a weak
> place of DRLVM and needs deep refactoring. I will try to figure out
> what we can do as a short term solution.
>
> Evgueni
>
> On 10/30/06, Tim Ellison <[EMAIL PROTECTED]> wrote:
> > Evgueni Brevnov wrote:
> > > BTW, here is two more interesting cases
> > >
> > > 1) Throw uncaught exception in a new thread. Both RI and DRLVM give 0
> > > in that case.

Interesting, I guess it does not depend on which Java thread
terminates last, only main is counted? Looks a bit inconsistent...


> > > 2) Call System.exit(123) in a new thread. Both RI and DRLVM give 123.
> > >
> > > Currently, DRLVM calls system _exit() at the end of System.exit().
> > > That's why DestroyJavaVM never returns. So the changes I proposed for
> > > the launcher do not help until DRLVM forcibly terminates the process.
> > > :-(
> > >
> > > Does IBMVME uses system calls to stop the proccess? Does DestroyJavaVM
> > > returns?
> >

Seems that explicit call to System.exit() and natural termination
(even by uncaught throwable) are different use cases and need separate
handling.


> > You cannot kill the entire process when destroying the VM.  Apps that
> > embed the VM will have a nasty surprise.


Is it the case for System.exit()/halt() ? Given the test result above,
I suspect RI and J9 just kill the process.


> >
> > Regards,
> > Tim
> >
> > > On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> > >> It seems we need to fix the launcher... which is common part for both
> > >> DRLVM and IBVME. I see the problem in the following code sequence...
> > >>
> > >> 
> > >>  (*jvm)->DetachCurrentThread(jvm);
> > >>  (*jvm)->DestroyJavaVM (jvm);
> > >> 
> > >>
> > >> The current thread has an uncaught exception raised before it calls
> > >> DetachCurrentThread. This exception is printed out by default uncaught
> > >> exception handler which is called upon thread detaching. So after
> > >> DetachCurrentThread completes we have no exception anymore... Moreover
> > >> we can not play with DetachCurrentThread and DestroyJavaVM return code
> > >> since it should not depend on uncaught exceptions. So I think the fix
> > >> should look like the following:
> > >>
> > >> 
> > >>   if (env->ExceptionOccured) {
> > >>rc = 1;
> > >>   }
> > >>  (*jvm)->DetachCurrentThread(jvm);
> > >>  (*jvm)->DestroyJavaVM (jvm);
> > >>  return rc;
> > >> 
> > >>
> > >> If there are no objections I'll come up with the patch soon...
> > >>
> > >> Thanks
> > >> Evgueni
> > >> On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> > >> > I'll try to see how to fix it for DRLVM
> > >> >
> > >> > Evgueni
> > >> >
> > >> > On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> > >> > > hmmm I got 0 for DRLVM on Win2003
> > >> > >
> > >> > > Evgueni
> > >> > >
> > >> > > On 10/30/06, Spark Shen <[EMAIL PROTECTED]> wrote:
> > >> > > > Then we may need to follow RI, since different error code fails
> > >> application
> > >> > > >
> > >> > > > Best regards
> > >> > > > Tony Wu ??:
> > >> > > > > Consider this class,
> > >> > > > > public class TestExeReturn {
> > >> > > > > public static void main(String[] args) throws Exception {
> > >> > > > > throw new Exception();
> > >> > > > > }
> > >> > > > > }
> > >> > > > >
> > >> > > > > when we run "java TestExeReturn" and "echo %errorlevel%", we
> > >> got 1 of
> > >> > > > > RI, -1 of DRLVM and 0 for IBMVM.
> > >> > > > >
> > >> > > > > a testcase of apache ant failed for this issue.
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Spark Shen
> > >> > > > China Software Development Lab, IBM
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
> > --
> >
> > Tim Ellison ([EMAIL PROTECTED])
> >
> >
>

Re: [DRLVM][GC] parallel compaction and wasted virtual space

On 10/29/06, Xiao-Feng Li <[EMAIL PROTECTED]> wrote:

Weldon, the problem is, there is no well-established parallel
compaction algorithms. So far the best known work are 1. SUN's work;
2. IBM's work and 3. Compressor. No one knows which one is the best
for different workloads. We have to identify one algorithm for
implementation, and at the moment Compressor looks to be the right
choice, or we write more than one compactors.

Let me ask a different question.  At the beginning of the email chain you
said:

If we have 1GB
physical memory, the JVM is ok for Compressor because the virtual
space is large enough to wast half; but if the phsical memory is >2GB,
Compressor may have a problem in 32bit machine: some of phsical mapped
space might be wasted.

Perhaps you mean to say that with the Compressor algorithm, the JVM's heap
is unable to grow beyond 2GB of virtual space.  The reason is because
Compressor double maps physical memory and thus can not grow beyond one half
of the virtual address space.  A 2GB Java heap can certainly run on 512MB of
physical memory.  The result will be unacceptable paging.   However, even if
a machine contains 4GB of physical memory, the Compressor still can not take
advantage of anything above 2GB of physical memory.

One question to ask is if restricting the size of the heap to less than 2GC
will meet the requirements for Harmony VM in the next 6 months or so.  If
the answer is no, an additional algorithm will need to be implemented.

Also,  the Comressor algorithm is only recently published.  Given that the
focus of Harmony is production quality JVM, there is a risk when
implementing any algorithm that is yet unproven in a production environment.

Thanks,

xiaofeng

On 10/30/06, Weldon Washburn <[EMAIL PROTECTED]> wrote:
> Since the Compressor algorithm was published only this year, perhaps it
> makes sense to consider it experimental.  Perhaps make it a compile time
> switch so that that folks focused on production quality VM don't trip on
> it.  Even assuming the implementation of the Compressor algorithm is bug
> free, there may be unforeseen performance problems that surface with
> different workloads.
>

--
Weldon Washburn
Intel Enterprise Solutions Software Division

Re: [drlvm] what's the difference between the structure VM_thread and HyThread?

2006-10-30 Thread Nikolay Kuznetsov


Hello,

this is legacy thread structure. You're right, there was no HyThread
at first, but VM_thread structure. Also different DRLVM modules used
to have dedicated fields in this structure. Further it was decided to
make threading module independent and make usage model of threading
module the same for all other modules(i.e use thread local storage for
module specific data). But since VM_thread structure usage was very
heavy it was also decided to leave all non threading data in VM_thread
and put it in TLS till better times (I mean, till this data will be
divided into module depended parts and also put into TLS under
different keys).

Nik.

On 10/30/06, Fan Bin <[EMAIL PROTECTED]> wrote:

Hi all,

I'm now reading the lastest version of drlvm. Now I have a question about the 
thread management module. I see that there are three kinds of control blocks: 
HyThread, JVMTIThread and VM_thread. What's the difference of their rols? 
Especially HyThread and VM_thread. As far as I know, there is no HyThread at 
first.

Thanks,
Fan Bin

Re: [build] Building on Eclipse - FYI


My concern is that if there is too much documentation and it's not well
structured then people won't be inclined to read any of it.  If 50% of the
"Getting Started With DRLVM" page is actually about Eclipse, my feeling is
that it won't be as useful for people who actually want to get started with
DRLVM than it would be if it was half the size and more concise.  In general
I think having Eclipse-related documentation on its own page (or
inline where it's relevant) is a good thing, I was just commenting on the
amount of seemingly unrelated Eclipse documentation that is currently
included in the DRLVM page (
http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started.html).
Just my 2p worth...

Thanks,

Sian


On 30/10/06, Geir Magnusson Jr. <[EMAIL PROTECTED]> wrote:




Sian January wrote:
> I'm actually not sure if that first page is up to date or not  (
>
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclipse.html
).
>
> I think it should be but I have not ever been able to get it to work
> myself.  Can anyone else confirm whether it is still in date or not?
>
> Also I'm just wondering whether we really need steps 5-14 of "Running an
> Application in Eclipse" or any of "Debugging an Application in Eclipse"?
> There doesn't seem to be anything Harmony-specific in those steps, it's
> just
> about using Eclipse.  I know Eclipse itself contains that kind of
> documentation in its help system so IMHO it seems like it could be a
little
> redundant to duplicate that on the Harmony website.
>

I think that if we have an eclipse document, and the info is accurate, I
say why not?

We're trying to suck people into the whirling vortex that is our little
project, and if it's someone that isn't an eclipse user, but was
intrigued by our ability to run it, etc, etc, it seem to be harmless.  I
assume that it won't change often, so the maintenance is minor?

geir

> Thanks,
>
> Sian
>
>
> On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>>
>> Sian,
>>
>> Thanks a lot for the detailed description of what I should do. I'll
>> follow your recommendations. :)
>>
>> I'll investigate what we have already written about Eclipse to avoid
>> redundant info and repeating pages.
>>
>> What I've just come across are:
>> 1) The page describing how to set up Eclipse to develop Java code in
>> Apache Harmony
>> [
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
>> pse.html];
>> 2) The sections "Running an Application in Eclipse" and "Debugging an
>> Application in Eclipse "in the Getting Started with DRL
>> [
file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
>> pse_Hello_world]
>>
>> Would be great, if you find a chance to check whether the
aforementioned
>> info is up-to-date :)
>>
>> Cheers,
>> Sveta
>> -Original Message-
>> From: Sian January [mailto:[EMAIL PROTECTED]
>> Sent: Friday, October 27, 2006 3:34 PM
>> To: harmony-dev@incubator.apache.org
>> Subject: Re: [build] Building on Eclipse - FYI
>>
>> Hi Sveta,
>>
>> That sounds like a great idea.  I think the two main things you need to
>> do
>> extra on Eclipse on Windows are as follows:
>>
>> 1. Make a copy of vsvars32.bat from your Visual Studio install
>> directory.
>> If you have chosen the defaults when installing you will find it in
>> "C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
>> Copy it
>> to somewhere convenient such as the desktop and add the following line
>> (just
>> below the last line that begins "@set..."):
>> start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
>> (where ... is the path to your eclipse installation).  NB. using
>> "-vmargs
>> -Xmx512M" is optional, but helpful to stop Eclipse running out of
>> memory.
>> Now just double click on this file to start Eclipse.
>>
>> 2. After you have started Eclipse and checked out Harmony from SVN,
copy
>> ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
>> the
>> Ant settings to include this jar (Window > Preferences > Ant > Runtime
>> then
>> select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
>> from
>> the org.apache.ant_1.6.5\lib directory).
>>
>> If you're happy to add that to the document that would be great.
>>
>> On Linux you will also need to do 2, but I'm not sure if there's an
>> equivalent to 1 or not.  Has anyone else tried building in Eclipse on
>> Linux?
>>
>> Thanks,
>>
>> Sian
>>
>>
>> On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]>
wrote:
>> >
>> > Folks,
>> >
>> > I see that we do need one more building doc describing Eclipse
>> > specifics.
>> > The main doc containing building instructions now is the "Getting
>> > Started for Contributors" page.
>> > [http://incubator.apache.org/harmony/quickhelp_contributors.html].
>> > My suggestion is to add one more section to it describing building
>> > instructions for Eclipse. How about that?
>> >
>> > If you need my help, I'll be glad to manage the doc creation:)
>> >
>> > Cheers,
>> > Sveta
>

[jira] Close these issues please

2006-10-30 Thread Denis Kishenko


I have two old verified and unassigned issues
http://issues.apache.org/jira/browse/HARMONY-1419
http://issues.apache.org/jira/browse/HARMONY-1175

Could anybody close them?

Thanks

Re: [drlvm] Class unloading support

2006-10-30 Thread Etienne Gagnon

>>> If I get it right, in case of automagic unloading, GC does all the job
>>> without a knowledge whether it collects a class, a classloader or
>>> whatever else.
>>> Perhaps I'm missing something, but to provide a callback on class
>>> unloading, the GC must know the semantic of the object being collected.

Am I wrong, or does this proposition imply collecting classes
independently from their class loader?  If this is the case, I have to
say that I disagree with the proposed approach.

The JVM spec says quite clearly:

 2.17.8 Unloading of Classes and Interfaces

 A class or interface may be unloaded if and only if its class loader is
 unreachable. The bootstrap class loader is always reachable; as a
 result, system classes may never be unloaded.

Just think about it.  One could take an instance "o" of a class C loaded
by L, call it (C,L), and call o.getClass().hashcode().  Store this
integer some where.  Then, "o" could die, and maybe (C,L) unloaded while
L is still reachable.  As L is still reachable, some code could do a
L.findClass("C").hashcode().  This will likely result in a different
hashcode, in full breach of the both the VM and API specifications.


In a related note, for memory management I highly encourage Drlvm to
look at Chapter 3 of http://sablevm.org/people/egagnon/gagnon-phd.pdf
that exposes a simple, yet very effective approach for managing
class-loader related memory (i.e. memory used to store internal class
data, vtables, jitted code) so that it can all be freed efficiently at
class-loader unloading time.

Etienne


-- 
Etienne M. Gagnon, Ph.D.http://www.info2.uqam.ca/~egagnon/
SableVM:   http://www.sablevm.org/
SableCC:   http://www.sablecc.org/


signature.asc
Description: OpenPGP digital signature

Re: [classlib][IBMVME]the return value of java.exe

2006-10-30 Thread Evgueni Brevnov


Hi,

https://issues.apache.org/jira/browse/HARMONY-2006 is created with
patches :-). I created two separate patches one for DRLVM another one
for Classlib. I don't know if IBMVME requires additional changes. Even
though DestroyJavaVM will not terminate the whole process now it still
doesn't clean up resources properly. I'm going to focus on that
problem...but it seems to take much more than one day :-)

Thanks
Evgueni.

On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:

Tim,

I agree, we shouldn't kill entire process. Unfortunately, it is how
current implementation does. Actually, shutdown process is a weak
place of DRLVM and needs deep refactoring. I will try to figure out
what we can do as a short term solution.

Evgueni

On 10/30/06, Tim Ellison <[EMAIL PROTECTED]> wrote:
> Evgueni Brevnov wrote:
> > BTW, here is two more interesting cases
> >
> > 1) Throw uncaught exception in a new thread. Both RI and DRLVM give 0
> > in that case.
> > 2) Call System.exit(123) in a new thread. Both RI and DRLVM give 123.
> >
> > Currently, DRLVM calls system _exit() at the end of System.exit().
> > That's why DestroyJavaVM never returns. So the changes I proposed for
> > the launcher do not help until DRLVM forcibly terminates the process.
> > :-(
> >
> > Does IBMVME uses system calls to stop the proccess? Does DestroyJavaVM
> > returns?
>
> You cannot kill the entire process when destroying the VM.  Apps that
> embed the VM will have a nasty surprise.
>
> Regards,
> Tim
>
> > On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> >> It seems we need to fix the launcher... which is common part for both
> >> DRLVM and IBVME. I see the problem in the following code sequence...
> >>
> >> 
> >>  (*jvm)->DetachCurrentThread(jvm);
> >>  (*jvm)->DestroyJavaVM (jvm);
> >> 
> >>
> >> The current thread has an uncaught exception raised before it calls
> >> DetachCurrentThread. This exception is printed out by default uncaught
> >> exception handler which is called upon thread detaching. So after
> >> DetachCurrentThread completes we have no exception anymore... Moreover
> >> we can not play with DetachCurrentThread and DestroyJavaVM return code
> >> since it should not depend on uncaught exceptions. So I think the fix
> >> should look like the following:
> >>
> >> 
> >>   if (env->ExceptionOccured) {
> >>rc = 1;
> >>   }
> >>  (*jvm)->DetachCurrentThread(jvm);
> >>  (*jvm)->DestroyJavaVM (jvm);
> >>  return rc;
> >> 
> >>
> >> If there are no objections I'll come up with the patch soon...
> >>
> >> Thanks
> >> Evgueni
> >> On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> >> > I'll try to see how to fix it for DRLVM
> >> >
> >> > Evgueni
> >> >
> >> > On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
> >> > > hmmm I got 0 for DRLVM on Win2003
> >> > >
> >> > > Evgueni
> >> > >
> >> > > On 10/30/06, Spark Shen <[EMAIL PROTECTED]> wrote:
> >> > > > Then we may need to follow RI, since different error code fails
> >> application
> >> > > >
> >> > > > Best regards
> >> > > > Tony Wu ??:
> >> > > > > Consider this class,
> >> > > > > public class TestExeReturn {
> >> > > > > public static void main(String[] args) throws Exception {
> >> > > > > throw new Exception();
> >> > > > > }
> >> > > > > }
> >> > > > >
> >> > > > > when we run "java TestExeReturn" and "echo %errorlevel%", we
> >> got 1 of
> >> > > > > RI, -1 of DRLVM and 0 for IBMVM.
> >> > > > >
> >> > > > > a testcase of apache ant failed for this issue.
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Spark Shen
> >> > > > China Software Development Lab, IBM
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
> --
>
> Tim Ellison ([EMAIL PROTECTED])
>
>

[drlvm] what's the difference between the structure VM_thread and HyThread?

2006-10-30 Thread Fan Bin

Hi all,

I'm now reading the lastest version of drlvm. Now I have a question about the 
thread management module. I see that there are three kinds of control blocks: 
HyThread, JVMTIThread and VM_thread. What's the difference of their rols? 
Especially HyThread and VM_thread. As far as I know, there is no HyThread at 
first.

Thanks,
Fan Bin

Re: [build] Building on Eclipse - FYI

Sian January wrote:

I'm actually not sure if that first page is up to date or not  (
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclipse.html). 

I think it should be but I have not ever been able to get it to work
myself.  Can anyone else confirm whether it is still in date or not?

Also I'm just wondering whether we really need steps 5-14 of "Running an
Application in Eclipse" or any of "Debugging an Application in Eclipse"?
There doesn't seem to be anything Harmony-specific in those steps, it's 
just

about using Eclipse.  I know Eclipse itself contains that kind of
documentation in its help system so IMHO it seems like it could be a little
redundant to duplicate that on the Harmony website.

I think that if we have an eclipse document, and the info is accurate, I 
say why not?

We're trying to suck people into the whirling vortex that is our little 
project, and if it's someone that isn't an eclipse user, but was 
intrigued by our ability to run it, etc, etc, it seem to be harmless.  I 
assume that it won't change often, so the maintenance is minor?

geir

Thanks,

Sian

On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:

Sian,

Thanks a lot for the detailed description of what I should do. I'll
follow your recommendations. :)

I'll investigate what we have already written about Eclipse to avoid
redundant info and repeating pages.

What I've just come across are:
1) The page describing how to set up Eclipse to develop Java code in
Apache Harmony
[http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
pse.html];
2) The sections "Running an Application in Eclipse" and "Debugging an
Application in Eclipse "in the Getting Started with DRL
[file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
pse_Hello_world]

Would be great, if you find a chance to check whether the aforementioned
info is up-to-date :)

Cheers,
Sveta
-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED]
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian

On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > I encountered a problem today buildi

Re: [drlvm][jit] How to override jit compilation?

2006-10-30 Thread Salikh Zakirov

Mikhail Fursov wrote:
> The solutions we have today:
> 
> 1) If you have only a few methods to be affected: create separate JIT
> instance without inliner and add method filters to EM configuration file.
> Check
> http://incubator.apache.org/harmony/subcomponents/drlvm/emguide.htmland
> *.emconf files for details.
> 
> 2) If you need to avoid inlining of special method in all Java methods you
> can use 'skip_methods' parameter of inliner pass. See opt.emconf file to
> check how to pass parameters to inliner.
> 
> 3) Replace all calls to your method with VMHelperCall in translator.
> Process
> it as direct calls in codegenerator. This solution was already proposed by
> Egor.

For the sake of completeness, there is one more solution

4) mark methods as 'native' in java sources and provide "native stub overrides" 
in
vmcore/src/jit/native_overrides.cpp.

Re: [drlvm][eclipse compiler] the test compiled with ECJ fails on drlvm

2006-10-30 Thread Evgueni Brevnov


Alexey,

I didn't mean to fix getEnclosing method impl itself. Yes the compiler
should provide such info. Sorry for confusion...

Evgueni

On 10/30/06, Alexey Varlamov <[EMAIL PROTECTED]> wrote:

2006/10/30, Nathan Beyer <[EMAIL PROTECTED]>:
> I see that one of the ECJ bugs has been fixed [1], but the other was
> closed awaiting more information [2], but may be resolved as well.
>
> The fix should be available to test from a nightly build of the
> Eclipse 3.3 stream. Would someone like to verify that the discussed
> issues are resolved?

Nathan, I've verified [2].

Elena, Evgueni,

The H-1931_ReflectExporter.patch is really the proper fix, thank you.
Regarding the question 3) of Evgueni's earlier post, I believe there
is nothing to fix in getEnclosingClass() impl: if compiler did not
care to provide such info, there is no reliable way to determine it
(using class name etc). And provided [1] is fixed in ECJ, Elena's
tests should pass.

--
Alexey

>
> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=162296
> [2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=162356



> On 10/26/06, Tim Ellison <[EMAIL PROTECTED]> wrote:
> > Elena Semukhina wrote:
> > > On 10/26/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> > >>
> > >> If this is a bug, have you logged an issue with Eclipse? If not,
> > >> please do so and post the bug URL here, so we can monitor it. You may
> > >> want to try compiling this with the latest ECJ JAR (3.3 nightly) to
> > >> see if it's still generating the same bytecode.
> > >
> > >
> > > Nathan, here is the bug URL:
> > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=162356
> > > I tried ecj.jar 3.3 and still was able to reproduce the bug.
> > >
> > > Considering that the RI can run this code fine, I'd be surprised if
> > >> this is considered a bug. I've been surprised before though. :)
> > >
> > >
> > > The test I submitted to Eclipse bugzilla fails on RI, IBM VM and DRLVM 
when
> > > compiled with ECJ and passes being compiled with javac.
> > >
> > > The fix submitted to H-1931 takes this bug into account and relies on the
> > > private modifier of a local class which is provided by Eclipse compiler
> > > (but
> > > not provided by javac). So the accessibility control algorithm takes
> > > different branches for the classes compiled with javac and ECJ for now.
> >
> > See also
> > https://bugs.eclipse.org/bugs/show_bug.cgi?id=162296
> >
> > which I have cross-referenced to Bug#162356.
> >
> > Regards,
> > Tim
> >
> > --
> >
> > Tim Ellison ([EMAIL PROTECTED])
> >
> >
>

Re: [classlib][beans]got a difference with RI

2006-10-30 Thread Alexei Zakharov


Hi Tony,

I agree that the current implementation handles digits incorrectly. So
+1 for fixing decapitalize() and following RI. Thanks for finding it.

Thanks,

2006/10/30, Tony Wu <[EMAIL PROTECTED]>:

this testcase passed on RI but failed on Harmony

public class TestBeans extends TestCase {
   public void test() {
   String expected = "a1";
   String actual = java.beans.Introspector.decapitalize("A1");
   assertEquals(expected, actual);
   }
}


spec does not mention the case containing a number, it says,
in the (unusual) special case when there is more than one character
and both the first and second characters are upper case, we leave it
alone.

But, IMO, we'd better follow RI here.


--
Alexei Zakharov,
Intel Enterprise Solutions Software Division

Re: [build] Building on Eclipse - FYI

I'm actually not sure if that first page is up to date or not  (
http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_eclipse.html).
I think it should be but I have not ever been able to get it to work
myself.  Can anyone else confirm whether it is still in date or not?

Also I'm just wondering whether we really need steps 5-14 of "Running an
Application in Eclipse" or any of "Debugging an Application in Eclipse"?
There doesn't seem to be anything Harmony-specific in those steps, it's just
about using Eclipse.  I know Eclipse itself contains that kind of
documentation in its help system so IMHO it seems like it could be a little
redundant to duplicate that on the Harmony website.

Thanks,

Sian

On 27/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:

Sian,

Thanks a lot for the detailed description of what I should do. I'll
follow your recommendations. :)

I'll investigate what we have already written about Eclipse to avoid
redundant info and repeating pages.

What I've just come across are:
1) The page describing how to set up Eclipse to develop Java code in
Apache Harmony
[http://incubator.apache.org/harmony/subcomponents/classlibrary/dev_ecli
pse.html];
2) The sections "Running an Application in Eclipse" and "Debugging an
Application in Eclipse "in the Getting Started with DRL
[file:///C:/SVN-23-10/docs/subcomponents/drlvm/getting_started.html#Ecli
pse_Hello_world]

Would be great, if you find a chance to check whether the aforementioned
info is up-to-date :)

Cheers,
Sveta
-Original Message-
From: Sian January [mailto:[EMAIL PROTECTED]
Sent: Friday, October 27, 2006 3:34 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [build] Building on Eclipse - FYI

Hi Sveta,

That sounds like a great idea.  I think the two main things you need to
do
extra on Eclipse on Windows are as follows:

1. Make a copy of vsvars32.bat from your Visual Studio install
directory.
If you have chosen the defaults when installing you will find it in
"C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools".
Copy it
to somewhere convenient such as the desktop and add the following line
(just
below the last line that begins "@set..."):
start C:\...\eclipse\eclipse.exe -vmargs -Xmx512M
(where ... is the path to your eclipse installation).  NB. using
"-vmargs
-Xmx512M" is optional, but helpful to stop Eclipse running out of
memory.
Now just double click on this file to start Eclipse.

2. After you have started Eclipse and checked out Harmony from SVN, copy
ecj_3.2.jar into ..\eclipse\plugins\org.apache.ant_1.6.5\lib and change
the
Ant settings to include this jar (Window > Preferences > Ant > Runtime
then
select 'Global Entries' then 'Add External Jars' and add ecj_3.2.jar
from
the org.apache.ant_1.6.5\lib directory).

If you're happy to add that to the document that would be great.

On Linux you will also need to do 2, but I'm not sure if there's an
equivalent to 1 or not.  Has anyone else tried building in Eclipse on
Linux?

Thanks,

Sian

On 26/10/06, Konovalova, Svetlana <[EMAIL PROTECTED]> wrote:
>
> Folks,
>
> I see that we do need one more building doc describing Eclipse
> specifics.
> The main doc containing building instructions now is the "Getting
> Started for Contributors" page.
> [http://incubator.apache.org/harmony/quickhelp_contributors.html].
> My suggestion is to add one more section to it describing building
> instructions for Eclipse. How about that?
>
> If you need my help, I'll be glad to manage the doc creation:)
>
> Cheers,
> Sveta
>
> -Original Message-
> From: Sian January [mailto:[EMAIL PROTECTED]
> Sent: Thursday, October 26, 2006 4:06 PM
> To: harmony-dev@incubator.apache.org
> Subject: Re: [build] Building on Eclipse - FYI
>
> Hi Nathan,
>
> Yes - I was trying to run the enitre build in Eclipse and this is
always
> my
> preferred method of building.  There is at least one other extra step
> apart
> from the one discussed in this thread so I think some additional
> documentation would be useful.
>
> Thanks,
>
> Sian
>
>
>
> On 25/10/06, Nathan Beyer <[EMAIL PROTECTED]> wrote:
> >
> > Are you referring to running the build scripts via Eclipse? Just
> > wanted to make sure I understand.
> >
> > Personally, I only import the module projects one at a time and run
> > the full builds outside of Eclipse, so I've never tried this.
Perhaps
> > some additional documentation on using Eclipse in this fashion. This
> > might be helpful additionally.
> >
> > On 10/24/06, Sian January <[EMAIL PROTECTED]> wrote:
> > > Hello,
> > >
> > > I encountered a problem today building on Eclipse, and I just
> thought
> > I'd
> > > post about it here in case anyone sees the same problem in the
> > future.  I
> > > was getting the error:
> > >
> > >
> > > BUILD FAILED
> > > *
> > >
> > > C:\eclipse32harmony\eclipse\workspace\Harmony\build.xml:108: The
> > following
> > > error occurred while executing this line:
> > >
> > >
> C:\eclipse32harmony\eclipse\workspace\Harmony\make\build

Re: [classlib][IBMVME]the return value of java.exe

2006-10-30 Thread Evgueni Brevnov


Tim,

I agree, we shouldn't kill entire process. Unfortunately, it is how
current implementation does. Actually, shutdown process is a weak
place of DRLVM and needs deep refactoring. I will try to figure out
what we can do as a short term solution.

Evgueni

On 10/30/06, Tim Ellison <[EMAIL PROTECTED]> wrote:

Evgueni Brevnov wrote:
> BTW, here is two more interesting cases
>
> 1) Throw uncaught exception in a new thread. Both RI and DRLVM give 0
> in that case.
> 2) Call System.exit(123) in a new thread. Both RI and DRLVM give 123.
>
> Currently, DRLVM calls system _exit() at the end of System.exit().
> That's why DestroyJavaVM never returns. So the changes I proposed for
> the launcher do not help until DRLVM forcibly terminates the process.
> :-(
>
> Does IBMVME uses system calls to stop the proccess? Does DestroyJavaVM
> returns?

You cannot kill the entire process when destroying the VM.  Apps that
embed the VM will have a nasty surprise.

Regards,
Tim

> On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
>> It seems we need to fix the launcher... which is common part for both
>> DRLVM and IBVME. I see the problem in the following code sequence...
>>
>> 
>>  (*jvm)->DetachCurrentThread(jvm);
>>  (*jvm)->DestroyJavaVM (jvm);
>> 
>>
>> The current thread has an uncaught exception raised before it calls
>> DetachCurrentThread. This exception is printed out by default uncaught
>> exception handler which is called upon thread detaching. So after
>> DetachCurrentThread completes we have no exception anymore... Moreover
>> we can not play with DetachCurrentThread and DestroyJavaVM return code
>> since it should not depend on uncaught exceptions. So I think the fix
>> should look like the following:
>>
>> 
>>   if (env->ExceptionOccured) {
>>rc = 1;
>>   }
>>  (*jvm)->DetachCurrentThread(jvm);
>>  (*jvm)->DestroyJavaVM (jvm);
>>  return rc;
>> 
>>
>> If there are no objections I'll come up with the patch soon...
>>
>> Thanks
>> Evgueni
>> On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
>> > I'll try to see how to fix it for DRLVM
>> >
>> > Evgueni
>> >
>> > On 10/30/06, Evgueni Brevnov <[EMAIL PROTECTED]> wrote:
>> > > hmmm I got 0 for DRLVM on Win2003
>> > >
>> > > Evgueni
>> > >
>> > > On 10/30/06, Spark Shen <[EMAIL PROTECTED]> wrote:
>> > > > Then we may need to follow RI, since different error code fails
>> application
>> > > >
>> > > > Best regards
>> > > > Tony Wu ??:
>> > > > > Consider this class,
>> > > > > public class TestExeReturn {
>> > > > > public static void main(String[] args) throws Exception {
>> > > > > throw new Exception();
>> > > > > }
>> > > > > }
>> > > > >
>> > > > > when we run "java TestExeReturn" and "echo %errorlevel%", we
>> got 1 of
>> > > > > RI, -1 of DRLVM and 0 for IBMVM.
>> > > > >
>> > > > > a testcase of apache ant failed for this issue.
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Spark Shen
>> > > > China Software Development Lab, IBM
>> > > >
>> > > >
>> > >
>> >
>>
>

--

Tim Ellison ([EMAIL PROTECTED])

Re: [drlvm][jit] How to override jit compilation?


Today I see several possible solutions. The complete solution with calling
arbitrary native methods from Java is not ready: we are just discussing it
in a separate thread. The first step of calling arbitrary native calls from
Java is VM helpers inlining framework. I'm going to commit it today and will
notify you about JIRA num. I hope it will serve as example.

The solutions we have today:

1) If you have only a few methods to be affected: create separate JIT
instance without inliner and add method filters to EM configuration file.
Check http://incubator.apache.org/harmony/subcomponents/drlvm/emguide.htmland
*.emconf files for details.

2) If you need to avoid inlining of special method in all Java methods you
can use 'skip_methods' parameter of inliner pass. See opt.emconf file to
check how to pass parameters to inliner.

3) Replace all calls to your method with VMHelperCall in translator. Process
it as direct calls in codegenerator. This solution was already proposed by
Egor.

--
Mikhail Fursov

Re: [drlvm][jit] How to override jit compilation?

2006-10-30 Thread Egor Pasko

On the 0x212 day of Apache Harmony Tonny Lau wrote:
> 30 Oct 2006 11:03:50 +0600, Egor Pasko <[EMAIL PROTECTED]>:
> >
> > On the 0x212 day of Apache Harmony Tonny Lau wrote:
> > > Hi,
> > >
> > > I want to override some specific java methods with native fast path
> > > implementations. So I try to override them in
> > > compile_do_compilation_jit()(vm/vmcore/src/jit/compile.cpp), that is,
> > > I add several entries in
> > >
> > _stub_override_entries_base[](vm/vmcore/src/util/ia32/base/compile_IA32.cpp),
> > > and lookup this table
> > > before invoke jit->compile_method_with_params().
> >
> > This mechanism is not used (AFAIR and AFAICanSeeNow). The alternative
> > way is to support your "magics" within each JIT. (Mikhail does things
> > like that in OPT just from JavaByteCodeTranslator.cpp (see
> > isMagicClass(...))
> 
> 
> Thanks! I'll look it.
> 
> > It works for JET, but failed when OPT recompile these method. Does the OPT
> > > go different path? If so, how can I override it? Does anyone can help
> > me?
> >
> > how does it fail? did you try it with -Xem:opt? (That's what I tend
> > to always ask about:)
> >
> > Just a guess: OPT expects some profile from JET, but cannot find any.
> 
> 
> I mean, the overriding succeeds when I use JET only, i.e., "java -
> Dem.properties=jet ...". If I use the default configuration "java ...", at
> the beginning, the overriding is successful because the code is compiled by
> JET. But later, when the EM invoke OPT to re-compile the hot code, the
> overriding does not work any longer. It fall back to the original java code.

yeah, that looks much like inlining :) hence, fixable in the inliner

> So, where do you think is the best place in OPT for the overriding
> code? 

I suppose overriding==your-custom-native-code?  Can your code be
written using ordinary (address) arithmetic and simple mem
allocations? 

If so, then the best method is to write it in Java using
*org.vmmagic.unboxed* package that Mikhail has just
implemented. Otherwise you can hack around
JavaByteCodeTranslator::invokevirtual and recognize your "magic"
methods there, replacing the method's invokation with your custom IR.

>And is there a common path for both JET and OPT?

No. Especially when you take OPT's inlining into
consideration. OPT can inline using only OPTs mechanisms that JET is
not aware of.

But *org.vmmagic.unboxed* can help you since it is supported both in
JET and OPT.

-- 
Egor Pasko, Intel Managed Runtime Division

Re: [classlib][IBMVME]the return value of java.exe