Re: [doc] What should be improved in DRLVM Doxygen documentation?
Andrey,
I'm trying to understand your metric and have got a question about
your it. Conssider the following file:
drlvm/trunk/vm/vmcore/include/Class.h. It has metric of 126. Does it
mean that this file has worse documentation than one of the folloiwng
files?
vm/gc_cc/src/timer.h (1) or vm/gc_cc/src/slot.h (1)
Seems like this metric likes big narrative text.
So do I :-)
Thanks!
On 11/16/06, Morozova, Nadezhda [EMAIL PROTECTED] wrote:
Andrey,
I personally like the metric. It does not always reflect the
proportional amount of documentation, but the files at the top of the
list (the worst files) indeed require attention :)
I guess the metric shows both undocumented files and those that are
pretty verbose but require more attention.
Suggestion: in addition to the big metric, have a more refined listing -
which ignores some warnings of minor importance. For example, a metric
that only reports undocumented members. What do you say?
Thank you,
Nadya Morozova
-Original Message-
From: Andrey Yakushev [mailto:[EMAIL PROTECTED]
Sent: Thursday, November 16, 2006 7:46 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [doc] What should be improved in DRLVM Doxygen
documentation?
Alexei's metric is interesting, but sometimes shows strange results
for pretty good docs, like for quite commented files:
0 verifier_8h.html
0 structVerifier_1_1vf__Graph.html
Seems like this metric likes big narrative text.
I also agree that comments quality could be estimated through Doxygen
warnings. I attempted to use such a metric for DRLVM.
I used generated DoxygenDrlvmLog.txt file and parser string:
---8--
cat DoxygenDrlvmLog.txt | grep Warning | awk -F : '{print $1}'
~/tmp/r ; for f in `cat ~/tmp/r | sort | uniq` ; do ( echo `cat
~/tmp/r | grep $f | wc -l` $f ) ; done | sort -n -r
---8--
The result is placed at
http://wiki.apache.org/harmony/DRLVM_Documentation_Quality_Doxygen_Warn
ing_
Rating
If it suits our needs we can think about regular testing.
Thanks,
Andrey
On 07 Nov 2006 14:23:20 +0600, Egor Pasko [EMAIL PROTECTED] wrote:
On the 0x216 day of Apache Harmony Alexei A. Fedotov wrote:
Nadya wrote,
we could check for required Doxygen tags in certain elements.
I wasn't asked, but cannot resist, sorry. You may achieve this
right
now
without additional coding. Doxygen warns about many problems you
describe, when you have the following option set.
# If WARN_IF_UNDOCUMENTED is set to YES, then doxygen will generate
warnings
# for undocumented members. If EXTRACT_ALL is set to YES then this
flag
will
# automatically be disabled.
WARN_IF_UNDOCUMENTED = YES
The resulting log consists of warning messages about different
problems.
My DoxygenDrlvmLog.txt, for example, contains the following one:
drlvm/trunk/vm/MMTk/ext/vm/HarmonyDRLVM/org/apache/HarmonyDRLVM/mm/mmtk
/
Scanning.java:47: Warning: The following parameters of
org::apache::HarmonyDRLVM::mm::mmtk::Scanning::precopyChildren(TraceLoc
a
l trace, ObjectReference object) are not documented:
parameter trace
does it make sense to convert warnings to quality metrics and put on
harmonytest.org (or even Wiki) regularly? It should encourage people
(like me) to document sources better. Or it is too much effort?
With best regards,
Alexei Fedotov,
Intel Java XML Engineering
-Original Message-
From: Morozova, Nadezhda [mailto:[EMAIL PROTECTED]
Sent: Friday, November 03, 2006 6:26 PM
To: harmony-dev@incubator.apache.org
Subject: RE: Re: [doc] What should be improved in DRLVM Doxygen
documentation?
Egor,
I agree with you on the idea of simplicity: documented vs.
non-documented.
An additional point: do you think we need/want to evaluate quality
of
comments? we could check for required Doxygen tags in certain
elements.
For example, a function is almost certain to include @param and
@return.
Surely, this is heuristics and does not solve all our problems.
But
the
Doxygen quality check sometimes shows that the file does have
comments,
but they are not processed properly by Doxygen - which results in
a
low
rating for an html file. Maybe this is a crazy idea - I'd be glad
to
know your opinion.
Thank you,
Nadya Morozova
-Original Message-
From: news [mailto:[EMAIL PROTECTED] On Behalf Of Egor Pasko
Sent: Friday, November 03, 2006 12:18 PM
To: harmony-dev@incubator.apache.org
Subject: Re: [doc] What should be improved in DRLVM Doxygen
documentation?
On the 0x216 day of Apache Harmony Alexei Fedotov wrote:
Egor,
Thank you for your interest.
We definitely need to improve our documentation. Necessity is not
a
real interest :)
Here is an algorithm:
1. Create a list of words from HTML files.
2. Merge a dictionary of all words used in documentation.
3. Remove a half of the most frequently used
Re: [drlvm][unit] 100% of class library tests pass
Pavel, The life started showing that you were correct. Today there were no report on http://harmonytest.org. Even if I would like to be a living notification, I couldn't. Vladimir, The thing which concerns me most is not an absence of results - I believe this is just a technical problem. For me the main problem is that without you there is no way to understand what happens. Can we made a process of reporting more open? For example, can we tune CC to send a notification on upload status? If there is a successful notification, then the file is uploaded successfully, and we need to ask Anton why it is not visible. Otherwise we'll know the problem is in CC. Thanks! On 11/16/06, Pavel Ozhdikhin [EMAIL PROTECTED] wrote: Sorry to say but it actually does not work until there is no notifications to the mailing list and no immediate reaction to the regressions. thanks, Pavel On 11/16/06, Fedotov, Alexei A [EMAIL PROTECTED] wrote: Pavel, All, Let me add one pro for the second approach: it works already. Vladimir's scripts daily upload test results to http://harmonytest.org. With best regards, Alexei Fedotov, Intel Java XML Engineering -Original Message- From: Tim Ellison [mailto:[EMAIL PROTECTED] Sent: Thursday, November 16, 2006 12:37 PM To: harmony-dev@incubator.apache.org Subject: Re: [drlvm][unit] 100% of class library tests pass Pavel Ozhdikhin wrote: We have to evolving systems - classlib and DRLVM. To check commits to classlib we need a stable DRLVM which can pass 100% of HUT. Otherwise it's impossible to use DRLVM for pre-commit testing - you never know whether your test fail because of your patch or due to latest changes in DRLVM. I remember the time when DRLVM and Jitrino actively evolved - for some time JIT had to use an older version of DRLVM which could pass all commit criteria because newer versions suffered from regressions. And finally we came to comon strict commit criteria which prevented regressions in both VM and JIT. To avoid regressions using DRLVM in classlib testing I see 3 possible solutions: 1. Use one fixed DRLVM version which can pass 100% HUT test. Update this version from time to time. Pros: + Less time to run DRLVM pre-commit tests + Classlib does not suffer from regressions in DRLVM Cons: - DRLVM will suffer from regressions - Classlib can not use the latest DRLVM - Need additional efforts to regain stability on DRLVM when we want to update the version for classlib testing 2. Add HUT to CruiseControl testing on DRLVM and rollback commits causing regressions Pros: + Less time to run DRLVM pre-commit tests + Classlib can use the latest DRLVM Cons: - Classlib can suffer from DRLVM regressions (time lag before rollback) - It is not always clear which commit caused a regression - Rollbacks are costly 3. Add HUT to the commit criteria for DRLVM Pros: + Classlib always can use the latest DRLVM + DRLVM has no regressions regarding to HUT Cons: - More time to run DRLVM pre-commit tests (I was told that HUT take 25 minutes running in single JVM mode) I think that preventing a problem is better than solving it afterwards. So, I personally would choose the 3rd approach, don't mind against the second and dislike the first one. Probably some combination of these is possible. While I appreciate the desire to keep things stable, I think it is unreasonable to ask developers to run the entire test suite each time. As we have seen in the classlib code, running targeted tests before commit and leaving the build machines to run continuous tests ensures that we are productive and are notified of breakages in time to easily back out a patch and re-evaluate. With the amount of machine time we have running harmony tests on different cpu's/os's/compilers/etc we are getting better coverage than any individual could be expected to provide. Which is a long way of saying I think option (2) above is best -- and relies on the bid machines letting us know if things break, and the commitment from all of us to go straight in and fix it. Regards, Tim -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. -- Thank you, Alexei
[drlvm][unit] New regression? org.apache.harmony.auth.tests.javax.security.auth.kerberos.serialization.KrbDelegationPermissionCollectionTest
Folks, Has anyone seen the following problem in the whole test run? I cannot reproduce the problem for standalone test. (SuSE 9) testcase classname=org.apache.harmony.auth.tests.javax.security.auth.kerberos.serialization.KrbDelegationPermissionCollectionTest name=testGolden time=0.047 error type=java.io.IOExceptionjava.io.IOException at java.io.IOException.lt;initgt;(IOException.java:35) at org.apache.harmony.luni.platform.OSFileSystem.close(OSFileSystem.java:203) at java.io.FileInputStream.close(FileInputStream.java:174) at org.apache.harmony.nio.internal.FileChannelImpl.implCloseChannel(FileChannelImpl.java:102) at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:98) at java.io.FileInputStream.close(FileInputStream.java:167) at java.io.FilterInputStream.close(FilterInputStream.java) at java.io.BufferedInputStream.close(BufferedInputStream.java:121) at java.io.FilterInputStream.close(FilterInputStream.java) at java.io.ObjectInputStream.close(ObjectInputStream.java) at org.apache.harmony.testframework.serialization.SerializationTest.getObjectFromStream(SerializationTest.java:201) at org.apache.harmony.testframework.serialization.SerializationTest.getObject(SerializationTest.java:520) at org.apache.harmony.testframework.serialization.SerializationTest.verifyGolden(SerializationTest.java:428) at org.apache.harmony.testframework.serialization.SerializationTest.verifyGolden(SerializationTest.java:402) at org.apache.harmony.testframework.serialization.SerializationTest.testGolden(SerializationTest.java:146) at java.lang.reflect.VMReflection.invokeMethod(Native Method) at org.apache.harmony.testframework.serialization.SerializationTest.runBare(SerializationTest.java:111) /error /testcase
Re: [general][testing] cruise control on the WinXP and SUSE linux
+1 On 11/15/06, Alexey Varlamov [EMAIL PROTECTED] wrote: Apparently this is a long awaited working (at least for drlvm testing), I'll be utterly surprised if somebody objects. Thank you! Here is My Big +1 :) 15.11.06, Vladimir Ivanov[EMAIL PROTECTED] написал(а): Hello everyone, I started cruise control (stored in the buildtest module with patch from issue 995) on the Windows XP and SUSE Linux boxes. Both machines are identical (1 CPU - P4*3GHz, 1GB RAM, 120Gb HDD). On each platform cruise control runs (as separate projects in СС terms, all settings have default values): - build of classlib module (target: 'rebuild'); - classlib tests on J9 VM (target 'test' in the classlib module); - build of drlvm module (target: 'build'); - vm tests (kernel+smoke+cunit, target: 'test' in the drlvm module); - classlib tests on DRL VM (target: 'test' in the classlib module with - Dtest.jre.home=drlvm); Is it OK to send my cruise control notifications to the harmony-commits list in order to notify about regressions? I suspect the notifications are not ideal but I will work on their improvement upon precedents (false alarms) and your feedback Thanks, Vladimir -- Thank you, Alexei
Re: [doc] What should be improved in DRLVM Doxygen documentation?
Mikhail, Sorry, I'm a bit slow. From my perspective making a documentation bundle from inter-component interfaces is an excellent idea! I started collecting interfaces from vm/include directory. Are there any other bundles a community is interested in? Being closer to the point, let me ask if you would be interested in an execution manager documentation bundle? Or do you prefer having it as a part of a JIT documentation bundle? If you vote for a separate bundle, which interface files should I pick for it? There is no vm/em/include directory, so I believe I should take a subset of vm/em/src/*.h files. Also, some inter-component interfaces also should be included, probably the following ones: drlvm/trunk/vm/include/em_intf.h drlvm/trunk/vm/include/open/ee_em_intf.h drlvm/trunk/vm/include/open/em.h drlvm/trunk/vm/include/open/em_profile_access.h drlvm/trunk/vm/include/open/em_vm.h Thanks! On 11/7/06, Mikhail Fursov [EMAIL PROTECTED] wrote: On 07 Nov 2006 19:08:35 +0600, Egor Pasko [EMAIL PROTECTED] wrote: Another problem: anyone who changes a document behaviour must update comments too :( excuse me, what is document behaviour? :) Read it as 'method/function behaviour'. I mean if you have good comments in file you can't submit a patch with code refactoring without update in comments. Does it become a new requirement? IMO we should start to collect metrics and add comments for intercomponent interfaces only (vm/include/*). -- Mikhail Fursov -- Thank you, Alexei
Re: [drlvm] [testing] Excluding commit tests until the problem is fixed
Pavel, you are correct. Rana, sorry for confusion. Both issues block passing class library unit tests. http://issues.apache.org/jira/browse/HARMONY-2070 [drlvm][thread] Unhandled exception in java.exe while java.util.jar module tests execution http://issues.apache.org/jira/browse/HARMONY-2073 [drlvm][unit] org.apache.harmony.beans.tests.java.beans.PersistenceDelegateTest I've used a debugger and caught an assert in exn_raise_by_name_internal for the second one. The first one contains three diffrent issues, and I cannot say where exactly the problem is. On 11/15/06, Pavel Afremov [EMAIL PROTECTED] wrote: As I understand Alexey means HARMONY-2073, but not HARMONY-2070. Alexei, is it correct? If not, could you clarify the point about exn_raise_by_name_internal in your initial letter, please? Pavel Afremov. On 11/8/06, Rana Dasgupta [EMAIL PROTECTED] wrote: OK thanks Pavel, I'll try the patch today. Rana On 11/8/06, Pavel Afremov [EMAIL PROTECTED] wrote: Hi Rana. I extend guard region as work around. It's only one way, which fix SOE on my SuSE Linux, without potential regression of your fix. On my Linux machine violation access signals happen one page before protected page on the stack. It's it. I ran all tests, and everything was OK. But strange misprint was fount in the new test. So I attach new fixed patch. Pavel Afremov. On 11/8/06, Rana Dasgupta [EMAIL PROTECTED] wrote: Though I tried several times, I could not repro 2070 or Alexey's specific problems. The test attached to 2018 repros, and that I think is enough. Pavel, 1. The patch looks good, but I could not apply and try it since my Linux box is down. 2. Did you run all tests ( smoke, cuint, kernel, and classlib )? Since this fully turns on lazy exceptions, we need to ensure that all tests pass, or at least have identical behaviour before and after the pacth. 3. Adding a finalizer based stack test to smoke is a good idea. 4. On Linux you extend the guard region up ( or down whatever ) by a page. Did you find a good reason for it, or is this just being careful? Rana On 11/7/06, Pavel Afremov [EMAIL PROTECTED] wrote: Rana, Everything is correct in you description, but it looks like that * HARMONY-2018* https://issues.apache.org/jira/browse/HARMONY-2018 should fix described bug. I think Alexei will have a chance to check it. Thank you. Pavel Afremov. -- Thank you, Alexei
Re: [drlvm] New regression: java.lang.ClassGenericsTest4
All, I wonder if we should apply zero regression policy to this change. Speaking simply, should this patch be reverted and kept for additional investigation? On 11/15/06, Alexey Varlamov [EMAIL PROTECTED] wrote: The guilty change is the following, which effectively turns on VM_LAZY_EXCEPTION support in exceptions_impl.cpp: Modified: incubator/harmony/enhanced/drlvm/trunk/vm/vmcore/src/exception/exceptions_impl.cpp URL: http://svn.apache.org/viewvc/incubator/harmony/enhanced/drlvm/trunk/vm/vmcore/src/exception/exceptions_impl.cpp?view=diffrev=475029r1=475028r2=475029 == --- incubator/harmony/enhanced/drlvm/trunk/vm/vmcore/src/exception/exceptions_impl.cpp (original) +++ incubator/harmony/enhanced/drlvm/trunk/vm/vmcore/src/exception/exceptions_impl.cpp Tue Nov 14 14:45:45 2006 @@ -26,6 +26,7 @@ #include Class.h #include classloader.h #include exceptions.h +#include exceptions_impl.h #include exceptions_jit.h #include exceptions_type.h #include environment.h So the problem most probably in exn_throw_by_class_internal() function, will look closer after lunch. 2006/11/15, Alexey Varlamov [EMAIL PROTECTED]: Heh, this regression is more interesting than it looked at first glance: JITs give the same output with identical stack trace, but test result is PASSED. By lucky chance I have older debug build at hand (svn = r474646) and it also spills this stacktrace to system err but status is PASSED for all execution engines. Looks like smth hase changed in exceptions processing for interpreter. 2006/11/15, Gregory Shimansky [EMAIL PROTECTED]: Hello Today a kernel tests which used to pass a day ago started to fail. It is java.lang.ClassGenericsTest4 (subtest test_3) [1]. I tried to revert some VM and classlib (since some important classes like URLClassLoader, Hashtable and TreeMap were changed recently) patches but no reversion helped. It may be a cumulative effect of the patches now makes the test fail. It fails somewhere deep inside of signature parser. The problem is also that class format parses uses antlr which makes the whole parsing quite complex. The failure happens on a BadSignatureTemplate class which is constructed from bytes, parsed and its method is invoked. Apparently it has a wrong signature (surprise :) ). The whole purpose of the test is not clear to me more surprising is that it works on RI (not on BEA). If someone could point to what may be wrong now with this test I would be very grateful. I also wonder why there is no InvocationTargetException which should be chained with the down the stack Exception. The test_3 method uses reflection to invoke a method BadSignatureTemplate.test_1 which in its turn threw an Exception in parser. Shouldn't there be an InvocationTargetException as the result of Method.invoke? Where can I look at BadSignatureTemplate.java, specifically line 13 which according to the stack called Class.getTypeParameters? [1] [EMAIL PROTECTED]: 10113$] ./lnx_ia32_gcc_debug/deploy/jre/bin/java -Xint -Xbootclasspath/a:./lnx_ia32_gcc_debug/semis/kernel.tests/classes:./make/tmp/junit.jar -Dtest.resource.path=./lnx_ia32_gcc_debug/semis/kernel.tests/resources junit.textui.TestRunner java.lang.ClassGenericsTest4 /lnx_ia32_gcc_debug/semis/kernel.tests/resources/org/apache/harmony/lang/generics/BadSignatureTemplate.class java.lang.Exception at org.apache.harmony.lang.reflect.parser.SignatureLexer2.nextToken(SignatureLexer2.java:420) at antlr.TokenBuffer.fill(TokenBuffer.java:69) at antlr.TokenBuffer.LA(TokenBuffer.java:80) at antlr.LLkParser.LA(LLkParser.java:52) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__SIMPLE_CLASS_TYPE_SIGNATURE(SignatureParser.java:1457) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__CLASS_TYPE_SIGNATURE_SUFFIXES(SignatureParser.java:1736) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__REFERENCE(SignatureParser.java:1408) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__CLASS_TYPE_SIGNATURE(SignatureParser.java:980) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__FIELD_TYPE_SIGNATURE(SignatureParser.java:844) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__BOUND(SignatureParser.java:1311) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__CLASS_OR_INTERFACE_BOUNDS(SignatureParser.java:1278) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__FORMAL_TYPE_PARAMETER(SignatureParser.java:1183) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__FORMAL_TYPE_PARAMETERS(SignatureParser.java:1152) at org.apache.harmony.lang.reflect.parser.SignatureParser.pr__FORMAL_TYPE_PARAMETERS_DECL(SignatureParser.java:951) at
Re: [drlvm] New regression: java.lang.ClassGenericsTest4
Guys,
This is a good discussion, and let me praise Alexey for the wonderful fix.
I'm a bit concerned about our accepptance checks. How this could be
that regression was missed by a committer and an engineer durring
acceptance test runs?
Bug comments showed that Gregory ran the tests before a commit. Do
tests report such problems clearly?
Thanks!
On 11/15/06, Pavel Afremov [EMAIL PROTECTED] wrote:
Oh. It's cool fix for my stupid bug.
Thanks for Alexey very much.
Pavel Afremov.
On 11/15/06, Alexey Varlamov [EMAIL PROTECTED] wrote:
Pardon for my English - a bit sleepy already...
2006/11/15, Alexey Varlamov [EMAIL PROTECTED]:
Err, what I found is really trivial bug. But it took quite a few time
to discover - seems today was not my day :(
Index: vm/vmcore/src/exception/exceptions_impl.cpp
===
--- vm/vmcore/src/exception/exceptions_impl.cpp (revision 475132)
+++ vm/vmcore/src/exception/exceptions_impl.cpp (working copy)
@@ -281,7 +281,7 @@
if (NULL != exception-exc_cause) {
tmn_suspend_disable_recursive();
-jthrowable exc_cause = oh_allocate_local_handle();
+exc_cause = oh_allocate_local_handle();
exc_cause-object = exception-exc_cause;
tmn_suspend_enable_recursive();
}
OK, we definitely need a regression test for this.
2006/11/15, Gregory Shimansky [EMAIL PROTECTED]:
Alexey Varlamov wrote:
2006/11/15, Alexey Varlamov [EMAIL PROTECTED]:
2006/11/15, Gregory Shimansky [EMAIL PROTECTED]:
Alexey Varlamov wrote:
The guilty change is the following, which effectively turns on
VM_LAZY_EXCEPTION support in exceptions_impl.cpp:
Well this is a patch from HARMONY-2018 which doesn't hide the
fact that
it enables lazy exceptions. Why shouldn't we enable them?
Gregory,
I've just re-read my posts and couldn't find anything critique or
offending - please don't take regressions too personal. I'm sure we
will be able to fix this one quite soon.
I wasn't offended in any way. I was just thinking that you know some
secret knowledge that lazy exceptions do not work and thus enabling
them
is wrong.
--
Gregory
--
Thank you,
Alexei
[drlvm][unit] 100% of class library tests pass
Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Thank you all! It takes 44 days for the great team we are. Thanks for your thoughtful, diligent work and deep inspiration. Kudos to you for the following (and not limited to this): * Alexey Varlamov and Elena for driving the whole process * Anton and Vladimir Ivanov for automating test runs * Geir and Gregory for checking and committing related DRLVM patches * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and committing related class library patches * Alexey Petrenko for becoming ICU expert and writing good JIRA issue resolution guidelines * Alexei Zakharov for resolving class library test issues * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for fixing class library and tests * Ivan, Egor, Mikhail Fursov, Nikolay Sidelnikov and Alexander Astapchuk for making execution engines work * Tatiana and Maxim for filing JIRA issues about test failures * Nikolay Kuznetsov for completing thread interruption handling and reverting consequences of park/unpark integration * Pavel Afremov for fixing exception handling * Boris Kuznetsov for intelligent fixing of security tests * Rana and Salikh for evaluating and discussing problems, reviewing and trying DRLVM patches * Pavel Pervov and Evgueni for help with DRLVM patches * Artem for discovering and fixing weird Windows* behavior * My wife for bringing hot tea to the computer during sleepless nights There are still open issues with reliability, multiprocessor and other special configurations, so the page http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. But this shouldn't prevent us from including class library testing into Harmony zero regression policy. What do you think? -- Thank you, Alexei
Re: [drlvm][unit] 100% of class library tests pass
Oops, I've missed: * Andrew Zhang for reviewing class library patches and helpful discussions On 11/16/06, Alexei Fedotov [EMAIL PROTECTED] wrote: Folks, According to http://harmonytest.org, today 100% of class library unit tests pass on DRLVM. Thank you all! It takes 44 days for the great team we are. Thanks for your thoughtful, diligent work and deep inspiration. Kudos to you for the following (and not limited to this): * Alexey Varlamov and Elena for driving the whole process * Anton and Vladimir Ivanov for automating test runs * Geir and Gregory for checking and committing related DRLVM patches * Paulex, Tim, Nathan, Stepan and Mikhail Loenko for checking and committing related class library patches * Alexey Petrenko for becoming ICU expert and writing good JIRA issue resolution guidelines * Alexei Zakharov for resolving class library test issues * Ilya Okomin, Denis Kishenko, Oleg Khaschansky and Alexey Ivanov for fixing class library and tests* Ivan, Egor, Mikhail Fursov, Nikolay Sidelnikov and Alexander Astapchuk for making execution engines work * Tatiana and Maxim for filing JIRA issues about test failures * Nikolay Kuznetsov for completing thread interruption handling and reverting consequences of park/unpark integration * Pavel Afremov for fixing exception handling * Boris Kuznetsov for intelligent fixing of security tests * Rana and Salikh for evaluating and discussing problems, reviewing and trying DRLVM patches * Pavel Pervov and Evgueni for help with DRLVM patches * Artem for discovering and fixing weird Windows* behavior * My wife for bringing hot tea to the computer during sleepless nights There are still open issues with reliability, multiprocessor and other special configurations, so the page http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM remains active. But this shouldn't prevent us from including class library testing into Harmony zero regression policy. What do you think? -- Thank you, Alexei -- Thank you, Alexei
Re: [jira] Patch available setting
Guys, I was trying to compare public announcements of new patches with private ones. The point of announcing new patches on harmony-dev@ is the following: anyone is welcome to verify a new patch. If no volunteers are willing to do so, then a committer or a bug submitter will do the job. Thanks, Alexei On 11/14/06, Salikh Zakirov [EMAIL PROTECTED] wrote: Alexey Petrenko wrote: So I think that *require* from issue creator to check the fix *before* not the best solution. Since noone on earth can prevent you from submitting your patch to a new JIRA and link it to the original one, this is not a *real* requirement. I guess this is just a hint to proper patch review / verification process. In any case, I do not see much of a problem. -- Thank you, Alexei
Re: [drlvm] drlvm and gdb and shared libraries
Let me point out the second section of Egor's manual - LD_LIBRARY_PATH is the only thing which should be additionally configured. On 13 Nov 2006 15:46:44 +0600, Egor Pasko [EMAIL PROTECTED] wrote: On the 0x220 day of Apache Harmony Geir Magnusson, Jr. wrote: I have a dumb question - I was playing today with a toy launcher for DRLVM working out some embedding issues, and for the life of me, I couldn't get dgb to ever let me break on anything in a shared library. I loaded the vm via dlopen() an dlsym(), and the code runs fine. But even build in debug, I couldn't ever specify a breakpoint in a shared lib even after it was loaded. has anyone run across this? you cannot enable a breakpoint until the library is loaded. Just wait until it is loaded. I wrote [1] to help with that. Hope it helps :) [1] http://wiki.apache.org/harmony/Debugging%20DRLVM%20with%20GDB%20on%20Linux -- Egor Pasko -- Thank you, Alexei
Re: [vm][classlib] hythr library
Andrey, Angela, If I understood a problem correctly Alexey asked to make a developer's life easier. It looks like now a developer should rebuild class library, then to rebuild DRLVM each time he wants to check his class library changes with DRLVM. Can we simplify the procedure? For example, if I don't change DRLVM, I shouldn't rebuild it each time to have correct files in jre/bin. * This is more conventional. * This allows people to use prebuilt DRLVM binaries. * Prebuilt binaries minimize astonishment for a class library developer. Alexey mentioned scripts as a quick solution. Is it possible to solve this problem on a build script level? Or should we change a launcher to reorder paths in LD_LIBRARY_PATH? Your design discussion about long term solution is quite interesting reading as well. Andrey said IBM should expose object layout. Let me speculate on it a bit. I really don't have any knowledge on J9, so any coincidence is not intentional. * Imagine that J9 is able to work with Sun's class library that was licensed by IBM from Sun for 10 years. * Imagine that class library has functions which access objects directly for performance reasons. Taking these two statements together I don't think we should count on exposing this object layout to the third party when this third party is a competing Open Source community. Thank you, Alexei On 11/13/06, Angela Lin [EMAIL PROTECTED] wrote: Hmm... Pre-Harmony, the IBM VM + classlib used the same thread library. When the classlib was contributed, I guess they forked the thread lib and changed the names of the functions. (I'm only speculating since I wasn't involved in that process.) hythr should be virtually identical to a subset of j9thr23.dll. I agree, the IBM VM + classlib should be using the same thread library. It shouldn't be hard to implement a redirector lib from hythr to j9thr. Would this obsolete the current classlib hythr implementation? Angela On 11/13/06, Andrey Chernyshev [EMAIL PROTECTED] wrote: On 11/13/06, Alexey Petrenko [EMAIL PROTECTED] wrote: Guys, is there any progress in making possible for IBM and DRL VM to use the same hythr library? I imagine they would have to share some more VM internals first, like GC or object layout interface, before they can migrate to a common threading library :) It is really a pain now to run class library tests on DRLVM now. And nearly impossible to easy switch VMs by launcher command line parameters. Since class library builds IBM version of hythr library and rewrites it in jre/bin directory. So you need to copy this library What if just disable copying of the hythr.dll into the deploy/jre/jdk/bin directory, may be harmony-vme could be providing the one? from DRL VM after each build. (Yes, I know how to write scripts :) If it is not possible for both VMs to use the same library probably we need to move these libraries to VM directories... +1. Seems like we don't have any VM at the moment which would be really using the hythr from classlib. Even IBM VME doesn't use the hythr. As Tim wrote earlier, VME is using it's own threading library called j9thr23 [1]. Does it differ a much from the hythr? Guess it might be favourable for the classlib + IBM VME stack as well if it was using a single thread library. Would it be possible for VME to include a straightforward hythr implementation which can delegate hythread_* calls to the j9thr23? Thanks, Andrey. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200606.mbox/[EMAIL PROTECTED] SY, Alexey -- Andrey Chernyshev Intel Enterprise Solutions Software Division -- Thank you, Alexei
Re: [drlvm][threading] Should hythread_monitor_init() aquire the monitor?
Evgueni, That was great. Artem, It's nice to see you online. Could you please check the last comments to http://issues.apache.org/jira/browse/HARMONY-1904 and decide what should we do about this issue?
Re: [testing][nio] 2 tests fail on the SUSE linux
Hello, Vladimir, I have checked JIRA and http://harmonytest.org. This looks like a new issue. I wonder if it could be a result of some recent fix? With best regards, Alexei On 11/13/06, Vladimir Ivanov [EMAIL PROTECTED] wrote: Hello everyone, today my cruise control failed on the linux SUSE box with 2 errors in the org.apache.harmony.nio.tests.java.nio.channels.SocketChannelTest test. Could somebody reproduce/ fix this issue? Thanks, Vlaidimir Tests testSocket_configureblocking and testCFII_ConnectAfterFinish_Server_NonBlock reports ERROR with similar output: The address is not available java.net.BindException: The address is not available at org.apache.harmony.luni.platform.OSNetworkSystem.socketBindImpl(Native Method) at org.apache.harmony.luni.platform.OSNetworkSystem.bind( OSNetworkSystem.java:126) at org.apache.harmony.luni.net.PlainSocketImpl.bind(PlainSocketImpl.java:161) at java.net.ServerSocket.init(ServerSocket.java:116) at java.net.ServerSocket.init(ServerSocket.java:72) at org.apache.harmony.nio.tests.java.nio.channels.SocketChannelTest.setUp( SocketChannelTest.java:77) -- Thank you, Alexei
Re: [jira] Patch available setting
Sian, This is a good catch. I have the following justification for 3. I think it is better process when a bug submitter validates a patch *before* commit to ensure that it is good. Also it validates the patch *after*. This worth to be requested via public alias. Why a bug submitter should do a pre-commit check? He has most interest in the bug resolution. Thanks! On 11/13/06, Sian January [EMAIL PROTECTED] wrote: Hi, I have just discovered that it's not possible for a contributor to set Patch available on a JIRA unless they reported it. (I'm not sure about committers as I'm not one...) I imagine this is to stop people coming in and editing other details on the JIRA, so I can see that it makes sense. My question is, what is the best thing to do if I attach a patch to a JIRA and I can't set Patch available? I can think of three alternatives at the moment: 1. Assume that the reporter will notice and set it themselves (or commit the fix if they're also a comitter) 2. E-mail the reporter privately 3. Post to the mailing list Or a fourth alternative would be a combination of the above where the person who contributed the patch waits a few days before doing either 2 or 3. Any thoughts on what would be best? Thanks, Sian -- Sian January IBM Java Technology Centre, UK -- Thank you, Alexei
Re: [drlvm][build] everyday SVN conflict on the version_svn_tag.h file
To get rid of conflicts I think we should remove this file from revision control. +1 On 11/10/06, Gregory Shimansky [EMAIL PROTECTED] wrote: On Saturday 11 November 2006 01:12 Alexei Fedotov wrote: generate this file as part of the build process? +1 for autogeneration of version_svn_tag.h It is kind of autogenerated already. To get rid of conflicts I think we should remove this file from revision control. On 11/10/06, Vladimir Ivanov [EMAIL PROTECTED] wrote: Hello everyone, When I do my morning SVN update of drlvm module I see the permanent conflict on the version_svn_tag.h file because this file is updated by build. Actually, it is not a big problem while it not breaks vm build. But it will be more convenient if these conflicts go out. Could we generate this file as part of the build process (it has only 4 strings)? May be some other solutions exists? I'm not too familiar with VM build so I'll be glad if somebody takes care about it :) -- Gregory Shimansky, Intel Middleware Products Division -- Thank you, Alexei
Re: [drlvm][jvmti][testing] I want to add JVMTI tests to drlvm commit checks
Gregory, I have checked the patch. I like it. Here are few notes. * When I used ant there were no for tag [1]. I used autogenerated ant files to run something in a loop. Solution with for takes less place and is more readable. * From my perspective 3 min timeout for a smoke test should be decreased. I think there should be no stress/reliaiblity loads during acceptance testing. The reason is simple: complex loads demonstrate unpredicable behavior and do not reveal problems with 100% accuracy, so the bad patch pass such acceptance tests sooner or later. * As for TestNG concern, I don't think we need to stick to the harness. When the time comes, we will change the harness painlessly. 1. http://mail-archives.apache.org/mod_mbox/ant-user/200410.mbox/[EMAIL PROTECTED] On 11/10/06, Gregory Shimansky [EMAIL PROTECTED] wrote: On Thursday 02 November 2006 23:24 Geir Magnusson Jr. wrote: That's an understatement. Don't feel bad. I've never seen anything like it before. The idea of generating ant scripts on teh fly is very unconventional. . You don't have enough cuts and bruises from working with the DRLVM build :) Ok I think I've come up with a reasonable compromise. I still used the whole system of converting XML and all the stuff. It does quite a lot of things in setup and init targets and using select is convenient. I don't know how to untangle all of the setup and not do a lot of duplication in ant scripting which I am not big expert in. But I managed to cut away the loop over components looking at how test target in build.xml is written. I've also converted smoke.test target to the same way because both jvmti and smoke tests are meant for a whole VM, not some component of it. This also made a weird bug go away when of smoke tests were built and run in some random subdirectory of semis instead of being in vm when they were ran separately as build smoke.test. Tests should be in their own subdirectories (main test inclusion/exclusion loop is done over them), main Java class for application has to be equal to have package and name equal to its subdirectory. Otherwise the build system won't know what to run. Other files may have any kind of names. I wrote one simple JVMTI test to start the suite. Other tests which I've experimented with I cannot submit because I didn't write them. I think they'll appear later from JIRAs like one in HARMONY-2143 which were submitted to ASF. Take a look at HARMONY-2151 and say what you think. If I don't get much opposition I'll commit the patch on this weekend. Don't shoot me. Writing even that much of Ant took a lot of time, beer and hair on my head. I said I am not an ant guru, didn't I? -- Gregory Shimansky, Intel Middleware Products Division -- Thank you, Alexei
Re: [testing] harmonytest.org new features
I like it! Thanks, Anton! On 11/10/06, Anton Luht [EMAIL PROTECTED] wrote: Hello, Yesterday I've deployed a new version to harmonytest.org New features include: - packages/suites/tests tree-like navigation (as in local JUnit html results).Tree navigation populated to old results, too. - the first page only includes 50 latest test runs, link to archive search added (includes search by uploader's name - request by Tony Wu) - filter diff results by test name (request by Alexei Fedotov) - detection of crashes - sometimes when suite crashes, there is an empty TEST-suite-name.xml file in the run results. If such file is found, all tests from this suite (detected from previous runs) are marked as crashed in this run. Bugs fixed: - duplicates in the results (if any) are merged (bug report by Tony Wu - test count on the site was bigger than real one) - there was a problem in uploading large files - ~ 5Mb - the results were not imported - playing with the configuration solved this problem - at least my test 11Mb file that broke the upload now uploads correctly. Please report bugs and send feature requests. -- Regards, Anton Luht, Intel Java XML Engineering -- Thank you, Alexei
Re: Re: [doc][drlvm] The document Getting started with DRL is outdated
Nadya, I think fixing a tutorial is a nice task for a newbie. I have filed a JIRA about it: http://issues.apache.org/jira/browse/HARMONY-2150 All, Please check that I've correctly added your corrections to the document. Alexei On 11/10/06, Morozova, Nadezhda [EMAIL PROTECTED] wrote: Alexei, Tutorials might be fine for mature projects, but I do not think ours is ready for a big flow of users yet, that would require a tutorial. So +1 for having a nice good tutorial ... one day. If there are volunteers to write the tutorial now, I'd be happy to help though. Thank you, Nadya Morozova -Original Message- From: Fedotov, Alexei A [mailto:[EMAIL PROTECTED] Sent: Friday, November 10, 2006 1:40 PM To: harmony-dev@incubator.apache.org Subject: RE: Re: [doc][drlvm] The document Getting started with DRL is outdated Guys, I like good tutorials. I learned VIM using a tutorial. I don't need the VIM tutorial any longer, but at the beginning it was useful. +1 for maintain Getting Started as is with minor changes Why Eclipse was chosen for the tutorial? Our goal was to use Harmony for Harmony development. I liked that idea. With best regards, Alexei Fedotov, Intel Java XML Engineering -Original Message- From: news [mailto:[EMAIL PROTECTED] On Behalf Of Egor Pasko Sent: Friday, November 10, 2006 1:29 PM To: harmony-dev@incubator.apache.org Subject: Re: [doc][drlvm] The document Getting started with DRL is outdated On the 0x21D day of Apache Harmony Pavel Ozhdikhin wrote: Nadya, One more proposal about Getting Started: let's remove all current content and write something like following: To the moment we got rid of all major differences from other Java implementations, so to use DRLVM you can just build it (here goes link to readme with build instructions) and run as any other Java VM. For commonly used command-line options please look into the Wiki page (link to Salikh's page or to the document). What do you think? 1 page to hold only 4 lines of text? :) Thanks, Pavel On 10 Nov 2006 14:29:59 +0600, Egor Pasko [EMAIL PROTECTED] wrote: On the 0x21D day of Apache Harmony Nadezhda Morozova wrote: Good day to you, Egor! evening, dark and snowy evening :) What do you say about the getting started doc? I expressed it recently. General idea is that Harmony operates near the same as other JSE implementations. Almost all specifics is in extra options which we started collecting on Wiki for an extra HOWTO-like page (BTW, thanks to Salikh for starting the page). I believe, it is time to remove the Getting Started. So, +1 to Pavel Ozhdikhin here. BTW, I asked my dad to look at the website. Ideas for improvement from him: 1) site-local search is useful for a beginner. Hm, Tomcat has it with links to google search. We can have something as soon as we get to the desired TLP :) 2) it is not obvious that site misprints/problems should be reported to the mailing list. Commercial websites have something like support/suggestions mailto. We can point mailto to the mailing list :) Thank you, Nadya Morozova -Original Message- From: news [mailto:[EMAIL PROTECTED] On Behalf Of Egor Pasko Sent: Friday, November 10, 2006 8:55 AM To: harmony-dev@incubator.apache.org Subject: Re: [doc][drlvm] The document Getting started with DRL is outdated On the 0x21D day of Apache Harmony Nadezhda Morozova wrote: Egor, I generally like the idea of improving navigation over the site - there's never too much of that. However, I am not sure whether we need yet another separate page for introductory/guidance info. I hope the starting page + the generic pages we have are mostly fine. However, adding a link here and there to lead site visitors. Getting started could be a more specific project-oriented page. There, you can tell people to go linkdownload/link, linkbuild/link the code. After which, they can start using it just as any other linkRI-compatible/link jdk. With the exceptions, see our linkwiki/link pages. To use the vm, readers might need to use the following options... If they want to read more on our VM, they can visit the linkcomponent/link page. If no website page contains an answer - they can read linkwiki faqa/link. .. or something like that :) Nadya, I really appreciate our efforts :) But this morning I woke up and looked the site structure with the eye of a beginner. And could not find any obvius flaws in the main structure. Left-side collection of links is in a very good shape, good for beginner-level navigation and contains almost all links you listed here. This was a really refreshing morning :) I'll ask some guys who are new to the project, how they feel about the site. And will report back, if I find something. Refreshing morning is over, now back to work.. Thank you, Nadya Morozova
Re: [drlvm][em64t] build fails
DRLVM build always requires prebuilt classlib... I believe ia32 classlib is ok until linking happens. On 11/11/06, Gregory Shimansky [EMAIL PROTECTED] wrote: Same for me. I saw this yesterday. It is because HARMONY-1558 has changed Class interfaces but didn't change x86_64 specific files. Now that I look at the error messages, I think it is quite easy to fix the build. I'll take care of weekend. I wonder how did you get to building drlvm if classlib didn't build for you because of -fpic problem? DRLVM build always requires prebuilt classlib... On Friday 10 November 2006 23:53 Stefano Mazzocchi wrote: I've managed to get everything in place for a DRLVM build.. it runs for a while and then it fails with these errors: [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/jit_lock_rt_s upport_em64t.cpp: In function 'void* rth_get_lil_monitor_enter()': [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/jit_lock_rt_s upport_em64t.cpp:127: error: 'managed_null' is not a member of 'Class' [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/jit_lock_rt_s upport_em64t.cpp: In function 'void* rth_get_lil_monitor_exit()': [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/jit_lock_rt_s upport_em64t.cpp:265: error: 'managed_null' is not a member of 'Class' [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/ini_em64t.cpp : In function 'void JIT_execute_method_default(void*, _jmethodID*, jvalue*, jvalue*)': [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/ini_em64t.cpp :201: error: 'struct Class' has no member named 'name' [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/ini_em64t.cpp :229: error: 'managed_null' is not a member of 'Class' [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/ini_em64t.cpp :326: error: 'managed_null' is not a member of 'Class' [cc] /home/stefano/src/harmony/drlvm/vm/vmcore/src/util/em64t/base/ini_em64t.cpp :359: error: 'struct Class' has no member named 'name' Suggestions? -- Gregory Shimansky, Intel Middleware Products Division -- Thank you, Alexei
Re: [drlvm][build] everyday SVN conflict on the version_svn_tag.h file
generate this file as part of the build process? +1 for autogeneration of version_svn_tag.h On 11/10/06, Vladimir Ivanov [EMAIL PROTECTED] wrote: Hello everyone, When I do my morning SVN update of drlvm module I see the permanent conflict on the version_svn_tag.h file because this file is updated by build. Actually, it is not a big problem while it not breaks vm build. But it will be more convenient if these conflicts go out. Could we generate this file as part of the build process (it has only 4 strings)? May be some other solutions exists? I'm not too familiar with VM build so I'll be glad if somebody takes care about it :) Thanks, Vladimir -- Thank you, Alexei
Re: [testing] Tests scores on http://harmonytest.org Was: [DRLVM] General stability
Anton, I like your approach to result comparison. 10% can be default value for some form field - anyone can change it if needed. As for test execution time reported by JUnit, it is applicable for stress tests as well if we gradually increase a load over time. Though using ttime field for stress tests and documentation rankings is not quite conventional. Probably more conventional would be to parse system-out![CDATA[]]/system-out for some metric. Does the site already parse TEST-* files? Thank you, Alexei On 11/10/06, Anton Luht [EMAIL PROTECTED] wrote: Hello, Alexei, I have related question. How can we improve http://harmonytest.org to make it possible to publish not just pass, fail, or error but numeric test scores? Easily - test results in JUnit reports have 'time' property - execution time in seconds. We can import and show them in the results. What else is needed? Maybe add something like 'show regressions' to the the 'compare runs' page? For example, show tests that increased execution time by more than 10% sorted by increase rate desc? -- Regards, Anton Luht, Intel Java XML Engineering -- Thank you, Alexei
Re: [doc][drlvm] The document Getting started with DRL is outdated
Nadya, I have failed to find web pages copyrighted by IBM on Apache using Google. Apache copyright usually contains no more than two lines. This page has fourty lines of legal staff. Is it really needed? Thanks! On 11/8/06, Morozova, Nadezhda [EMAIL PROTECTED] wrote: All, I'd like to share everyone's grief at the sight of outdated Getting Started document. However, I'd not hurry to eliminate the page as such. We might reconsider some of its contents, change structure, and update individual bits, but please think carefully before removing the page. I think Getting Started (as the title shows) is aimed to help a newbie work with our vm. I know that many primarily interested in other things - conformance, architecture, internal specifics. However, we should also think how the vm is used. AFAIK, Getting started is now the *only* doc that tries to show how to use our vm. You tell people how to download and build, but almost nothing about how to run and configure (with the exception of EM/JIT). My suggestion would be to think of what you want to tell people about usage - with or without eclipse specifics. And store this info on the page. I know it is hard - and I offer my help and support in this burdensome initiative. Any thoughts? i might be inobjective and emotional :) Thank you, Nadya Morozova -Original Message- From: Mikhail Fursov [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 08, 2006 6:03 PM To: harmony-dev@incubator.apache.org Subject: Re: [doc][drlvm] The document Getting started with DRL is outdated It's not a hard to write a documenation once, it's hard to support it :) More problems: 1) -Xem options are obsolete. 2) -Xjit options are also obsolete. 3) Do we really need this page today? AFAIU users expect Harmony VM is able to run the same apps as RI.. ? On 11/8/06, Pavel Ozhdikhin [EMAIL PROTECTED] wrote: Hello all, I've read through the Getting Started with DRL http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started. html document on the Harmony web and found it completely outdated, for example: - the term DRL is used instead of DRLVM - eclipse.bat and eclipse.sh are obsolete - we don't need them anymore to run Eclipse. It can be started with DRLVM the same way as with any other VM. - We don't need to set PATH and LD_LIBRARY_PATH anymore, at least on Windows/MSVC - ij was renamed to java We took a big step to unification with other Java VMs and now we don't need anything specific to run Eclipse, for example. After removing all irrelevant info the document would contain only the list of command-line options. I think we can move this list to a separate document (Wiki, Developer's Guide?) and remove the Getting Started itself. Any opinions? Thanks, Pavel http://incubator.apache.org/harmony/subcomponents/drlvm/getting_started. html -- Mikhail Fursov -- Thank you, Alexei
Re: [jira] Good issue resolution guideline (was: [classlib]volunteer to supply patches for old JIRAs)
All, I have added a patch to http://issues.apache.org/jira/browse/HARMONY-2077 [doc] Good issue resolution guideline, though some things are rephrased though. On 11/6/06, Fedotov, Alexei A [EMAIL PROTECTED] wrote: Alexey, Thank you for your answer. Yesterday I tried to prepare an appropriate patch for get-involved.html but faced several things I couldn't resolve myself. I didn't criticize your text - I was thinking how to fit it into get-involved.html. That is why I raised all these questions. I still have one question which is important, to my mind. You wrote, DRLVM is not the only Harmony VM. I think that developer can choose which VM to use. 1. As Geir says, we are small community, and we need to keep ourselves concentrated. Can we give a recommendation to concentrate on one specific VM assuming that VM which is shipped with Harmony 1.0? 2. What is the status of DRLVM smoke tests? As any tests written in java they have a dependency from a class library. Do you think they should be skipped while testing the patch? With best regards, Alexei Fedotov, Intel Java XML Engineering -Original Message- From: Alexey Petrenko [mailto:[EMAIL PROTECTED] Sent: Monday, November 06, 2006 2:03 PM To: harmony-dev@incubator.apache.org Subject: Re: [jira] Good issue resolution guideline (was: [classlib]volunteer to supply patches for old JIRAs) 2006/11/6, Fedotov, Alexei A [EMAIL PROTECTED]: Alexey, I started looking into this thread. Are guidelines on the web site already? I failed to find them. No, this guideline is not published yet. I'll do this in the nearest future. I thought about adding them to get-involved.html. Here are few problems I've noticed: First of all As you know patches are always welcome in Harmony :) 1. The detail level of your instruction is somewhat different from get-involved.html. This text is formal, while the text on the page is quite informal. I do not see problem here. 2. If a guy has a test in a separate file, should he produce a patch from such file and submit the patch? According to guidelines, the answer is yes. Do you mean in a new file? Yes, a new file should be provided as a patch too. Why not? According to my common sense patches are good when you modify existing files. Yes, they are good for modified files. And they also can be used for adding and removing files. 3. Why it is not suggested for an issue reporter to try localizing a buggy module I think that points 2 and 3 are saying this. or even fixing the problem? Nobody said that an issue reporter can not be a resolution provider. 4. Item 4. of issue reporter guidelines doesn't contain enough details for my taste. I believe links should be descriptive: Link the issue to previous posts, JIRAs, RI bugs, etc. Probably your sentence is better. Need to think. 5. The item 2.1 of resolution provider guidelines contains too many details to my mind. Here should be something like a following text (for all roles): a. Update JIRA once per day reporting your progress. b. Always keep the community posted. What for? Why do you want to have DAILY posts on ALL the working issues? I think that we do not need to change anything here since comunity has agreed on the list of notifications it wants to see. And all these cases are described there. 6. The item 2.4 refers to class library build.xml as a main build.xml. Patches are welcome :) 7. You do not specify which unit tests should pass and which VM should be used. Since the changes could affect DRLVM, I would say that all unit tests should pass for DRLVM unless the fall into exclude list. DRLVM is not the only Harmony VM. I think that developer can choose which VM to use. 8. I believe a verification stage should happen before committer commits a patch - this would save his time if the issue doesn't resolve the problem. Yes, and nobody argue with this. SY, Alexey -Original Message- From: Alexey Petrenko [mailto:[EMAIL PROTECTED] Sent: Thursday, September 28, 2006 5:02 PM To: harmony-dev@incubator.apache.org Subject: Re: [jira] Good issue resolution guideline (was: [classlib]volunteer to supply patches for old JIRAs) :) Yes, I'll prepare a patch. 2006/9/28, Geir Magnusson Jr. [EMAIL PROTECTED]: On Sep 28, 2006, at 3:21 AM, Alexey Petrenko wrote: Guys, Since there is no additional comments on this guideline... Let's put it somewhere. We got two options: Harmony site and wiki. I prefer wiki because it will be easy to edit it and I can put it there myself :) And if you put in a patch for website, we can get it there too :) if you put in wiki, I'm going to take and put on site, so maybe save us some effort? (ok, save me the effort...) geir Thoughts? SY, Alexey 2006/9/26, Alexey Petrenko [EMAIL PROTECTED]: I've combined all the comments again. And here is the last version. I hope... :) === cut === Preface This guideline covers
Re: [general] Interesting discoveries playing around with japitools
Alexey wrote, As far as I remember TCK does not check for signature compatibility. http://jcp.org/en/resources/tdk reads: Signature Test tool The Signature Test tool verifies that a Java technology implementation undergoing compatibility testing and its reference APIs are mutually compatible as defined in Chapter 13, Binary Compatibility, of The Java Language Specification. The tool automates this verification process with a signature test algorithm that compares the API under test with a reference API. On 11/5/06, Alexey Petrenko [EMAIL PROTECTED] wrote: Really interesting results! :) It seems that japitools are not used for certification :) As far as I remember TCK does not check for signature compatibility. It's just a number (huge number) of tests... SY, Alexey 2006/11/5, Stefano Mazzocchi [EMAIL PROTECTED]: Being a sucker for statistics and charts, I've decided to look into japitools myself and regenerate the graph of API coverage progress of harmony. In doing so, I started to familiarize myself with the Japitools and I found a few interesting discoveries: the latest version of the three freely available certified java 5 implementations (Sun's, BEA's and IBM's) are *NOT* 100% compatible. So, here's what I did: 1) download the three JVMs 2) download japitools, tar xzf it and cd japitools 3) type: ./bin/japize as $name packages \ /path/to/jvms/$name/jre/lib/*.jar \ +java +javax +org -org.apache -org.ietf and substitute $name with the JVM name 4) you'll obtain three $name.japi.gz files (the binary representation of your API footprint) 5) then type japicompat -s $original.japi.gz $test.japi.gz where original is the JVM that you consider the reference and $test is the one that you want to test. Here are the (a little surprising, I must say) results: -- sun 1.5 vs. bea1.5 --- 99.99% good, 0% missing java.awt.peer: method java.awt.peer.WindowPeer.updateFocusableWindowState(): missing in /home/stefano/data/japi/ibm1.5 -- sun 1.5 vs. ibm1.5 --- Total: 99.93% good, 0% bad, 0.06% missing java.awt.peer: Missing method java.awt.peer.WindowPeer.updateFocusableWindowState(): missing in /home/stefano/data/japi/ibm1.5 javax.xml.datatype: Bad field javax.xml.datatype.DatatypeFactory.DATATYPEFACTORY_IMPLEMENTATION_CLASS: constant [com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/sun1.5, but constant [org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/ibm1.5 org.omg.stub.javax.management.remote.rmi: Missing class org.omg.stub.javax.management.remote.rmi._RMIConnectionImpl_Tie: missing in /home/stefano/data/japi/ibm1.5 class org.omg.stub.javax.management.remote.rmi._RMIServerImpl_Tie: missing in /home/stefano/data/japi/ibm1.5 org.w3c.dom.html: Missing method org.w3c.dom.html.HTMLFrameElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLIFrameElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLObjectElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLOptionElement.setSelected(boolean): missing in /home/stefano/data/japi/ibm1.5 - o - There is one method that both Bea and IBM don't implement in awt.peer.WindowPeer and apparently, Sun doesn't implement it either.. it's just a stub! Weird. [see more at http://forums.java.net/jive/thread.jspa?messageID=167137] the differences in datatype factory is plausible and the fact that a stub RMI class is missing is not that big of a deal. It's weird though, that the DOM in IBM's is different than the DOM in Sun's... I guess not that many people use the HTML DOM in java or they would have got that ;-) The really crazy things start to happen if you flip things around and you consider the 'clean room rewrites' as the reference implementations: -- bea1.5 vs. sun1.5 Total: 99.98% good, 0.01% missing javax.xml.namespace: Missing class javax.xml.namespace.QName: missing in /home/stefano/data/japi/sun1.5 Uh? Sun forgot to ship the QName class or this is a japitools bug? googling up shows the class in the java1.5 docs so it's more likely it's a bug in japitools http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/namespace/QName.html Gets more interesting with IBM: -- ibm1.5 vs. sun1.5 Total: 99.77% good, 0% bad, 0.22% missing java.lang: Missing method java.lang.StringBuilder.append(java.lang.StringBuilder): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.capacity(): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.charAt(int): missing in /home/stefano/data/japi/sun1.5 method
Re: Brush up Harmony pages
The first link at the page http://incubator.apache.org/harmony/subcomponents/classlibrary/index.html to Getting Started For Contributors is now incorrect - it contains http://incubator.apache.org/docs/quickhelp_contributors.html instead of http://incubator.apache.org/harmony/quickhelp_contributors.html Sorry, I haven't prepeared a patch for this issue.
Re: [drlvm] dynamic object layout
Weldon, I agree with you that it is nearly impossible to achieve stability for a branch under active development. From the other side, adding new features is fun, and also has a reason behind it. If we strive for a complete implementation of J2SE, we cannot avoid this type of activity. So my suggestion is to create separate branches for new features which could be merged into the main branch when mature enough to achieve an appropriate level of stability. What do you think? Alexei On 11/3/06, Weldon Washburn [EMAIL PROTECTED] wrote: Salikh, I glanced at the patch. What you propose below looks reasonable. I really don't see any other way to do it and still get usable performance. All, My only worry is disturbing highly critical code like object layout. Given that this JIRA has been open a long time, I guess its OK to apply the patch. At some point, we need to stop adding functionality and focus on stability. On 11/3/06, Salikh Zakirov [EMAIL PROTECTED] wrote: Hi, I am currently continuing to work on improving JVMTI Heap Iteration (HARMONY-1635), particularly, tagging objects. The use case that I've heard of is tagging *all* objects for the purpose of memory profiling. According to what I've heard it causes 60x slowdown on Sun's VM. However, the initial tags implementation that I've uploaded to HARMONY-1635 is far worse, as it uses linear search for get/set tag operations. (* for those who didn't read JVMTI spec, tags are jlong (8 byte integer) values, which can be attached to arbitrary objects in get/set manner *) The alternative approach I came up with is to use (mostly) constant time algorithms for get/set operations, is to store a tag pointer in each object. Storing tag itself in an object is not an option, as JVMTI requires to send OBJECT_FREE events with tags for each reclaimed objects, and this information would not be available if the tag would be reclaimed together with the object. However, since the general consensus was that increasing object header is highly undesired, I've tried to implement the _conditional_ increase in object header. Additional object header field is allocated in case JVMTI Agent has requested can_tag_objects capability. The modified object layout I used is as follows: +---+ | VTable pointer | +---+ | lockword | +---+ | [array length] | +---+ | [tag pointer] | +---+ |[padding] | +---+ | fields or elements| | ... | +---+ Where [array length] is only present in array objects, [tag pointer] is only present when can_tag_capability has been enabled at startup [padding] is only present in arrays of longs and doubles for natural 8-byte alignment. VTable pointer is really uint32 offset on em64t/x86_64 and ipf/ia64. The only difference with current object layout is introduction of tag pointer field. I've modified gc_cc to take the changed dynamic object layout into account, and surprisingly it took only one modification: * use VM function vector_first_element_offset_unboxed() instead of hardcoding first array element offset. This is done once for each class done at loading stage, and gc_cc caches this offset for later uses. I've experimented with putting tag pointer at fixed location before array length, but it looks expensive, as it will add one more read to GC array scanning, and we obviously do not want optimize at the expense of common case. The latest version of the patch is attached to HARMONY-1635 ( heap-iteration-optimized.patch), I would appreciate any comments and concerns. -- Weldon Washburn Intel Enterprise Solutions Software Division -- Thank you, Alexei
Re: [doc] What should be improved in DRLVM Doxygen documentation?
Egor, Thank you for your interest. Here is an algorithm: 1. Create a list of words from HTML files. 2. Merge a dictionary of all words used in documentation. 3. Remove a half of the most frequently used words from the dictionary - I believe they do not add much sense. 4. Remove misspelled words (including identifiers) from the dictionary. 5. Give a page +1 for each rare, correctly spelled word according to the dictionary. 6. Divide to the total number of words on the page. I've collected nice RFEs from your letter. Most of them make me think and I like them. a. Update an ASF block comment b. Improve readability. Some things are really easy - like removing awk and rewriting most things in perl. Others are a bit more complex - I targeted script performance when created auto-generated perl script. Also, initial algorithm was a bit more complex - different words had a different cost based on their popularity. c. Use junit test output format to integrate with http://harmonytest.org. I believe I need a feature request for that site as well - we need some way to import performance-like rankings to the site. d. I will think of parsing sources. But I don't think we need to maintain both scripts. The generic rule is simple - improve your .h and .java files - .cpp files don't count. I suggest better to link .html files to contributors. Thank you for ideas. I will certainly update the script. I just want to wait a bit - many scripts die just because people are not interested to run them a second time. Also, if anyone suggest any changes in algorithm or any other RFEs, I want to implement them all at once. Nadya, could you please point us a good documentation file so we can use it as a pattern? -- Alexei
Re: [doc] What should be improved in DRLVM Doxygen documentation?
Egor, I pursued a requirement to compare HTML documents of a different nature. For this requirement comparing a number of documented items vs not documented ones doesn't work. For source files your metrics is the best. Let me comment on unique words. The goal was to subtract words like Copyright, Apache, Software, Foundation which appear on every page. I debugged my metrics using the following pattern: I compared pages visually and than compared their metrics. If the results of comparison were different, I fixed a metric. Thanks, Alexei On 03 Nov 2006 15:17:38 +0600, Egor Pasko [EMAIL PROTECTED] wrote: On the 0x216 day of Apache Harmony Alexei Fedotov wrote: Egor, Thank you for your interest. We definitely need to improve our documentation. Necessity is not a real interest :) Here is an algorithm: 1. Create a list of words from HTML files. 2. Merge a dictionary of all words used in documentation. 3. Remove a half of the most frequently used words from the dictionary - I believe they do not add much sense. 4. Remove misspelled words (including identifiers) from the dictionary. 5. Give a page +1 for each rare, correctly spelled word according to the dictionary. 6. Divide to the total number of words on the page. hm, strange heuristic. More unique correctly spelled words is beneficial. It does not give a clue on the overall quality of documentation, which is rather confusing.. I thought of something more natural. Number of documented items vs. number of non-documented. Plus a penalty to the relative number of misspelled words. I've collected nice RFEs from your letter. Most of them make me think and I like them. a. Update an ASF block comment b. Improve readability. Some things are really easy - like removing awk and rewriting most things in perl. Others are a bit more complex - I targeted script performance when created auto-generated perl script. Also, initial algorithm was a bit more complex - different words had a different cost based on their popularity. c. Use junit test output format to integrate with http://harmonytest.org. I believe I need a feature request for that site as well - we need some way to import performance-like rankings to the site. Yes, I thought of the RFE to harmonytest. At least, put the doc items on a separate page from the build items. d. I will think of parsing sources. But I don't think we need to maintain both scripts. The generic rule is simple - improve your .h and .java files - .cpp files don't count. I suggest better to link .html files to contributors. can you calculate a list of relevant filenames from a doc page? give filename +1 for each documented item, give a -1 for each undocumented, divide on the number of items. Is it easy to implement? Maybe doxygen has some features to assist this? Thank you for ideas. I will certainly update the script. I just want to wait a bit - many scripts die just because people are not interested to run them a second time. Also, if anyone suggest any changes in algorithm or any other RFEs, I want to implement them all at once. Nadya, could you please point us a good documentation file so we can use it as a pattern? -- Egor Pasko
Re: [doc] What should be improved in DRLVM Doxygen documentation?
Nadya, You asked good questions. Here are few answers: 1. Grouping of results is implied by documentation grouping. Scripts can process any documentation bundle, so if one creates a smaller or specific bundle, the list will be shorter, or more specific.Creating several documentation bundles in different directories makes their comparison an easy task - I can do this comparison. 2. Personally I like @page tags and package.html files. I appreciate Salikh's efforts of creating wonderful technical descriptions - I referred to them as masterpieces. I also remember that you asked me to create a narrative section for a component manager few months ago. All Doxygen documentation will be on the web site. Why these narrative sections shouldn't be evaluated? 3. I don't think the rating of pages such as a list of functions should be neglected. Any .html page which can be viewed by a user should be readable. That is a reason why I parse .html files in the script, not sources. 4. I believe establishing connection between .html files and source files can be automated. I don't know how to write a short script for that, because sometimes .html page depends from several source files, and vice versa. 5. You can imagine the following pie chart from the data: 2680 pages of 2922 (91%) are not good and should be revised. 6. Today the community discussed removing GC V4. This would immediately decrease GC documentation size. It would also decrease a number of well documented pages on garbage collection, since new GCs don't have as much comments in their code as old GC V4. Thank you for nice catches, Alexei On 11/2/06, Morozova, Nadezhda [EMAIL PROTECTED] wrote: Wow! Alexei, great start! ... and so many pages marked with 0 rank. I really appreciate your effort - it sets me back on earth and to work. I hope this rating would also make owners of code more ambitious, and would inspire them to write more/better comments to get a better rating :) Question on output measurement: can we rate source files or code elements (structure, functions, etc) instead of html files? My concerns: - Many html files are autogenerated, their rating is N/A. examples: todo.html, functions_vars_0x68.html (listing of links to functions in alphabetical order - there are many pages like that) - Some results are duplicated, because each struct/function has an individual rating + rating of the file/group reference it belongs to. - Some files have a high rating (see the top candidate, for example), but it's generated from comments marked with @page. These don't belong to specific code, but create a narrative section. Evaluating these is complex, and perhaps, should not be done. My personal preference would be to move such generic explanations to component docs on the website and reserve Doxygen docs to API reference as much as possible (this is a subject for further discussion). - the listing of files is SO LONG... grouping them by file/component/type or otherwise organizing the output would make the whole rating more readable. I mean, from the current version, I can only make out the leaders (not files even, individual bits of them), and understand that the majority have 0 rating. This has its instructional impact, but I cannot see the areas where we are the best - bearable - worst, or see the approx distribution of powers... missing that info leaves me without direction on what to do. Question on data presentation: do you think we can have some post processing of the raw data that your script produces - to see the big picture? We have some metrics: graphics, pie charts, anything. This would instantly show the most important conclusions. I could do such metrics and post them regularly on Wiki. If anybody says they need such kind of info, I'd volunteer to help. Thank you, Nadya Morozova -Original Message- From: Fedotov, Alexei A [mailto:[EMAIL PROTECTED] Sent: Thursday, November 02, 2006 11:33 PM To: harmony-dev@incubator.apache.org Subject: [doc] What should be improved in DRLVM Doxygen documentation? Nadya, All, I have ranked the quality of Doxygen-generated DRLVM documentation and posted it to the following Wiki page: http://wiki.apache.org/harmony/DRLVM_Documentation_Quality All are welcome to check masterpieces of our documentation. All volunteers are welcome to improve page ranks by editing comments in DRLVM sources. With best regards, Alexei Fedotov, Intel Java XML Engineering
