[admin] Automated Monthly Reminder
This is a monthly automated mailing to the Apache Harmony dev list. The Apache Harmony project welcomes the participation and input of anyone interested in open source Java SE, the goal of our project. For more information about the Apache Harmony project, please see the project website. http://incubator.apache.org/harmony/ The terms of use for the Harmony mail lists can be found here : http://incubator.apache.org/harmony/mailing.html and they are This forum has been created for public communication about projects of The Apache Software Foundation (the Foundation), a Delaware nonprofit corporation classified as a public charity under 501(c)(3). All communication intentionally submitted to the Foundation on this forum is considered a Contribution to the Foundation unless otherwise noted in the communication. The terms and conditions that apply to your Contributions are defined by either a contributor license agreement (CLA) signed by you and/or your employer or, if no such CLA is on file at the Foundation, by the terms and conditions of Contributions as defined by the Apache License, Version 2.0. Note : * If you do not wish your post to be a Contribution, we would prefer that you do not post it. However, in the event that you do, please mark as NOT A CONTRIBUTION at the top of the posting. * Do not post any code that is not your original work, or code that you do not have clear authorization to contribute. * Do not engage in detailed discussion of any implementation that you have been exposed to unless such implementation is available to everyone under an open source license or is your own implementation. * Under no circumstances will any committer accept code for inclusion in our SVN repository contributed on the mailing list unless it is from an Authorized Contributor, as defined here. If you have any questions, please do not hesitate to ask on either the dev list. If there are any questions that are of a private or sensitive nature, please send them to [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] Trying to catch patternset errors earlier
On 28 September 2006 at 14:58, Alexey Petrenko [EMAIL PROTECTED] wrote:
2006/9/28, Mark Hindess [EMAIL PROTECTED]:
On 28 September 2006 at 14:30, Alexey Petrenko [EMAIL PROTECTED]
om wrote:
I think that it will be better to add another target to build for
this check.
Because of two reasons:
1. It is unclear that clean is also checks something
This simple check can only happen part way through the clean target -
after the modules have done their cleaning and before the top-level
cleanup is done.
I see your point.
But let's review the sitatuation from another viewpoint...
What do we need in general?
We need up-to-date list of all packages in the specified module. Right?
If so we can create simple script to update patternset files... Or
even create them at build time. But will cost us build time itself :)
That's a reasonable point. I had a look at what it would take to get
rid of the patternsets completely. It's quite easy for some modules.
For example, for accessibility, we can just replace the classes
fileset with:
fileset id=classes dir=${hy.build}
or
present targetdir=${hy.accessibility.src.main.java} /
present targetdir=${hy.accessibility.src.main.java}
mapper type=regexp
from=^(.*?)(\$$[^/\\\.]*)?\.class$$
to=\1.java/
/present
/or
/fileset
Which basically says include files if they are also present in
the source directory - typically resource files - or if they are
.class files and have an obviously-named source file in the source
directory[0].
This works for most modules. The present tags are duplicated for
modules with platform source directories.
However, it doesn't work very well for awt, beans, jndi, luni, and
security. Why? Because they have some source files defining two
classes. For instance, java.awt.ContextImpl defined in:
modules/awt/src/main/java/common/java/awt/Component.java
We could hardcode these special case - there are only a few. Or perhaps
we could split them in to distinct source files?
I have some patches ready to make this change but I've not committed
anything.
Personally I think getting rid of the patternsets would be a good idea
since it reduces the coupling - although really the coupling is only
strictly necessary for luni and security.
It is important to note that even if we do something like this, leaving
the catch-all check in the clean target still helps stop problems - if
people forget to add the hardcoded special cases or if we split them
people add new source defining two classes in one file.
Regards,
Mark.
[0] The simpler 'mapper type=glob from=*.class to=*.java'
doesn't work because it misses inner classes.
2006/9/28, Mark Hindess [EMAIL PROTECTED]:
On 28 September 2006 at 11:07, Tim Ellison [EMAIL PROTECTED] wrot
e:
Sounds reasonable. The alternative is to not make clean fail, just
print the warning and tidy up the remainder. That may be too easy to
ignore though.
Yes. I considered that and had the same concern. Even if I wrote the
code to print the warning, I think I'd ignore it since it would scroll
too quickly off the top of my screen at the beginning of the build.
-Mark.
Regards,
Tim
Mark Hindess wrote:
Yesterday, while looking at something unrelated, I noticed that som
e
of the patternsets that are used to select the jars for the classli
b
modules were not up to date with the result that some classes would
be
missing from the resulting jars[0].
While it makes me slightly uneasy having a clean target that might
fail
,
it turns out that this is one place where it is quite easy to check
whether the patternsets are out of date. (Because if after the mod
ules
have cleaned classes in their patternsets there are still files lef
t
over then something is being create that isn't accounted for by the
patternsets.)
So the clean target will now fail with a message like (tested
by reverting yesterdays change to the sound patternset):
Built files still exist after module clean targets have run. Thi
s
probably means that one or more patternsets are incomplete. The
remaining files are:
/classlib/build/classes/org/apache/harmony/sound/utils/ProviderSe
rvic
e.cl
ass
I'm sure there are other ways to solve this problem but this seemed
lik
e
a sensible quick fix to help catch these problems a little sooner[1
].
Regards,
Mark.
[0] This might explain some of the awt/swing test failures so perha
ps
it is worth checking the exclude lists again?
[1] Though I guess since we clean at the beginning of a build (not
the
end) then we might still find them in the build after the one that
caused the
[classlib][launcher] should we get rid of one of the HyPortLibrary function tables in DRLVM?
All, Using windows debugger, I see native/launcher/shared/main.c::invocation() receive an incoming argument that looks to be a DRLVM version of HyPortLibrary with all the functions zeroed out. Does anyone else see this?? Passing a HyPortLibrary with the function ptrs nulled out is not the primary concern. At worst, this will cause a sigsegv and should be straight forward to debug. The big concern is accidentally using the classlib/HyPortLibrary function ptr table when DRLVM Threading Manager APIs are intended. This could cause all sorts of strange deadlocks. I have looked at the code to prove or disprove that the two HyPortLibraries are being confused. So far, no luck. There are too many layers to get to the bottom of this quickly. Does anyone know the answer to the above question? If not, should I open a JIRA on this issue? -- Weldon Washburn Intel Middleware Products Division
Re: [classlib][auth]LoginContext should always invoke the LoginModules?
On 9/29/06, Paulex Yang wrote:
Hi, all
I'm not a security expert, so please correct me if I miss something. I
found some different behavior of Harmony and RI on
javax.security.auth.login.LoginContext, the testcase[1] shows the
difference.
Actually I tried to create the event sequence like below:
1. create LoginContext with some Subject
2. LoginContext.login() and return successfully
3. Modify Subject's content to make it invalid(one Principal's name
here, maybe passwd/username/servername in more general case)
Hi, Paulex
LoginContext doesn't verify Subject's contents - all requred info is
obtained with callback handler and passed to login modules. And login
modules check whether password/username are valid or not.
4. LoginContext.login() again
In RI, the second login() invocation really tried to invoke the relative
LoginModule.login() and then failed to login with the modified Subject,
but in Harmony, both invocations succeed. I consider RI's behavior is
more reasonable.
After a rough look of LoginContext implementation, I found the cause may
be the Ln. 275
private void loginImpl() throws LoginException {
if (loggedIn) {
return;
}
}
I think the spec. statement: A LoginContext should not be used to
authenticate more than one Subject. was taken too strict: reusing
LoginContext object to get the same set of credentials seemed odd.
But if RI let LoginContext object to be reusable then it makes sense doing
the same.
Thanks,
Stepan.
Seems Harmony won't invoke the LoginModule.login() again only if the
login ever succeeds. If I comment out these lines, the test below passes
happily. Any ideas on this issue?
[1]
public class LoginContextTest extends TestCase {
private static final String VALID_NAME = name1;
private static final String INVALID_NAME = name2;
public void testLogin() throws Exception{
MyPrincipal pri = new MyPrincipal();
HashSet set = new HashSet();
set.add(pri);
Subject sub = new Subject(false, set, new HashSet(), new
HashSet());
Configuration.setConfiguration(new MyConfig());
LoginContext context = new LoginContext(moduleName, sub);
context.login();
pri.name = INVALID_NAME;
try{
context.login();
fail(Should throw LoginException);
}catch(LoginException e){
}
}
static class MyConfig extends Configuration{
AppConfigurationEntry[] entries = new
AppConfigurationEntry[]{new
AppConfigurationEntry(MyModule.class.getName(),
LoginModuleControlFlag.REQUIRED, new HashMap())};
public AppConfigurationEntry[] getAppConfigurationEntry(String
name) {
return entries;
}
public void refresh() {
}
}
public static class MyModule implements LoginModule{
Subject sub;
public void MyModule(){
}
public boolean abort() throws LoginException {
return false;
}
public boolean commit() throws LoginException {
return true;
}
public void initialize(Subject arg0, CallbackHandler arg1,
MapString, ? arg2, MapString, ? arg3) {
sub = arg0;
}
public boolean login() throws LoginException {
Principal[] pris = sub.getPrincipals().toArray(new
Principal[0]);
return VALID_NAME.equals(pris[0].getName());
}
public boolean logout() throws LoginException {
return false;
}
}
public static class MyPrincipal implements Principal{
public String name = VALID_NAME;
public String getName() {
return name;
}
public String toString(){
return name;
}
};
}
--
Paulex Yang
China Software Development Lab
IBM
--
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib][auth]LoginContext should always invoke the LoginModules?
On 9/30/06, Paulex Yang wrote:
Paulex Yang wrote:
Hi, all
I'm not a security expert, so please correct me if I miss something. I
found some different behavior of Harmony and RI on
javax.security.auth.login.LoginContext, the testcase[1] shows the
difference.
Actually I tried to create the event sequence like below:
1. create LoginContext with some Subject
2. LoginContext.login() and return successfully
3. Modify Subject's content to make it invalid(one Principal's name
here, maybe passwd/username/servername in more general case)
4. LoginContext.login() again
In RI, the second login() invocation really tried to invoke the
relative LoginModule.login() and then failed to login with the
modified Subject, but in Harmony, both invocations succeed. I consider
RI's behavior is more reasonable.
After a rough look of LoginContext implementation, I found the cause
may be the Ln. 275
private void loginImpl() throws LoginException {
if (loggedIn) {
return;
}
}
Seems Harmony won't invoke the LoginModule.login() again only if the
login ever succeeds. If I comment out these lines, the test below
passes happily. Any ideas on this issue?
I've removed these lines at revision r451557 with regression test,
please shout if anyone thinks the update harmful for some reason.
I'll look into to verify if the update is harmless.
Thanks,
Stepan.
[1]
public class LoginContextTest extends TestCase {
private static final String VALID_NAME = name1;
private static final String INVALID_NAME = name2;
public void testLogin() throws Exception{
MyPrincipal pri = new MyPrincipal();
HashSet set = new HashSet();
set.add(pri);
Subject sub = new Subject(false, set, new HashSet(), new
HashSet());
Configuration.setConfiguration(new MyConfig());
LoginContext context = new LoginContext(moduleName, sub);
context.login();
pri.name = INVALID_NAME;
try{
context.login();
fail(Should throw LoginException);
}catch(LoginException e){
}
} static class MyConfig extends Configuration{
AppConfigurationEntry[] entries = new
AppConfigurationEntry[]{new
AppConfigurationEntry(MyModule.class.getName(),
LoginModuleControlFlag.REQUIRED, new HashMap())};
public AppConfigurationEntry[] getAppConfigurationEntry(String
name) {
return entries;
}
public void refresh() {
}
}
public static class MyModule implements LoginModule{
Subject sub;
public void MyModule(){
}
public boolean abort() throws LoginException {
return false;
}
public boolean commit() throws LoginException {
return true;
}
public void initialize(Subject arg0, CallbackHandler arg1,
MapString, ? arg2, MapString, ? arg3) {
sub = arg0;
}
public boolean login() throws LoginException {
Principal[] pris = sub.getPrincipals().toArray(new
Principal[0]);
return VALID_NAME.equals(pris[0].getName());
}
public boolean logout() throws LoginException {
return false;
}
}
public static class MyPrincipal implements Principal{
public String name = VALID_NAME;
public String getName() {
return name;
}
public String toString(){
return name;
}
};
}
--
Paulex Yang
China Software Development Lab
IBM
--
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [drlvm] HARMONY-1582 - invocation API for DRLVM
On 9/30/06, Andrey Chernyshev [EMAIL PROTECTED] wrote: On 9/27/06, Evgueni Brevnov [EMAIL PROTECTED] wrote: On 9/27/06, Andrey Chernyshev [EMAIL PROTECTED] wrote: On 9/26/06, Evgueni Brevnov [EMAIL PROTECTED] wrote: Alexey Varlamov [26/Sep/06 05:11 AM] Evgueni, thank you, quite impressive work! Unfortunately patch is so huge it is hard to understand at once. I understand your concern (about reformatting)... I feel very-very uncomfortable when I study purely formatted code. It always hard to understand some one's code especially if it is hard to read. That was not my intention to fix indenting I just can't study such code and reformat it on the fly Sorry, I don't know how to switch my patch to the original formatting. And what really bothers, is that about half of it is just reformatting :( Can't we really go without white space changes and renaming of parameters\local vars??? Its definitely much less than the half of the patch. Kind request: could you please describe shortly what is done in TM - which essential changes enhancements? I think a lot of people are interested in answer to this question. Do you? Let me get a break and I will come up with details Right, it seems like invocation API patch does extensive changes to the TM design and interfaces. Hopefully you can provide us with a summary why it was done so. Sure! In the meantime, there are a few things I'd like to understand first (I didn't look through the whole patch yet though): (1) Why did you have to add JavaVM to jthread_create and jthread_attach functions? I couldn't see the use for JavaVM in the TM code, except putting it into TLS which can always be done outside of TM if one wishes to do so. Ideally, we would have TM to know as much less about the rest of VM as possible. Mainly because I'm looking forward to have multi VM in a process address space. So I tried to do everything having this in mind. Take jthread_attach function. Originally it had JNIEnv * as its first argument. According to JNI spec JNIEnv * pointer is valid in the current thread only! Before thread is not attached it doesn't have its environment. So it can be a parameter to jthread_attach function. To be honest I go iniline with original design here VM calls jthread_attach to attach threadjthread_attach calls vm_attach (vm_jthread_attach now). We may think about changing that to the following. VM attaches thread to it self by means of vm_jthread_attach and then passes created JNIEnv * pointer to jthread_attach. What do you think? I would expect TM be responsible for launching threads, and VM be responsible for creating JNIEnv for each new thread. VM may have a method like create_JNIEnv() which TM might be using while building a new Java thread (or attaching the existing native one). We can extend the existing vm_attach() function for that purpose for now, such that it would be returning new JNIenv's. I did exactly like you suggest here :-) Please look at vm_jthread_atach. (2) What is the purpose of adding hythread_lib_create and hythread_lib_destroy functions? I guess we already have hythread_init/hythread_shutdown for the similar purpose... Multi VM again. Each VM should have its own instance of the library. Looking into the patch, it really seems like you just added a convenience method which does an allocation of the hythread_lib_t structure with help of APR. How does that help to solve the multiVM issue? hythread_lib was a variable in vm_main in the past, but what prevents you from just making it a part of the appropriate JavaVM struct? Yes, I did it. If you look at Global_Env structure you will find it have a pointer to portlib. (3) One more lock is added - hythread_lib_lock. How is that differ from the hythread_global_lock that we already have? Each extra lock to the system may add more possibilities for deadlocks, as well as can negatively impact the scalability (unless some of the existing locks are split). hythread_lib_lock acquires exactly the same lock as hythread_global_lock. Probably I miss something but we need to be compatible with IBM threading library now. This library has such function. That's why I added it. Sounds right? OK, then why don't suggest just to rename the hythread_global_lock to the hythread_lib_lock if you feel this is that important? I guess this must be more safe if we were introducing one more global lock to the system. I agree with you. There is no need to keep both. I just left former one for not introducing aaditional changes in this patch. Do you suggest to include that in this patch or do it separatly? (4) Instead of adding daemon attribute to hythread_create, I think the better approach would be to move the daemon threads count logic out of the hythread level - this level is assumed to know nothing about daemon threads in Java, the
Re: [drlvm] HARMONY-1582 - invocation API for DRLVM CHECKPOINT
On 9/29/06, Andrey Chernyshev [EMAIL PROTECTED] wrote: On 9/29/06, Evgueni Brevnov [EMAIL PROTECTED] wrote: Artem, Thank you for your feedback find my inlined. On 9/29/06, Artem Aliev [EMAIL PROTECTED] wrote: Evgueni, I got most of your changes, but still disagree with all hythread/jthread interface changes. Could leave interface unchanged. See following possible solutions, that could solve the same problems without interface changes. 1) daemon attribute is a java specific. (Andrey mentioned this already). Could you please move it back. to the jthread layer. It is better to rename hythread_wait_for_all_nondaemon_threads() to jthread_wait_for_all_nondaemon_threads(). Ok, I see no problems to move daemon to java layer. In that case: 1) I will move hythread_wait_for_all_nondaemon_threads() from thread_init.c to one which implements java layer. 2) I will move daemon field from HyThread structure. Agree? Sounds good to me. OK, will do that. 2) JavaVM could be retrieved from JNIEnv by jni_get_java_vm(). So let the jthread_create() and others to use JNIEnv (that is passed from java native method). The vm_attach could get old JNI env and create new one for the new thread. The first JNIEnv is created in CreateVM call and could be passed to the first thread at startup. No, no, no. I completely disagree with that!!! Why do you like JNIEnv but JavaVM. Why do you think that passing JavaVM instead of JNIEnv makes TM less modular? I don't see any difference here It seems for me like a big big hack to grab JNIEnv from another thread and pass it to jthread_attach to perform operations in the current thread. TM needs to know JNIEnv, mainly for managing the references to objects, throwing exceptions and calling run() method of a new thread. JNI spec proposes that JNIEnv is valid within the given thread, this is why TM holds the JNIEnv pointer at the moment. This is why TM likes the JNIEnv. Having the JNIEnv, one is able to get JavaVM but not vise versa. This is why TM doesn't like the JavaVM :) I see your point. Only one note this is true for already attached threads... I agree with you that there is a design flaw that the JNIEnv is copied from the parent thread to a child thread during thread creation. I think it could be resolved via vm_attach() hook - with this event, VM might tell the TM what the JNIEnv pointer for new thread should be. I think you did that by extending the vm_attach() call with the JNIEnv** argument. What is not completely clear is, why do you have to pass the JavaVM forth and back, once from VM to TM, and then back from TM to VM again? If you need to know in jthread_attach, which particular VM vm_attach() event is coming from, you could have vm_attach like vm_attach(JNIEnv* currentThreadEnv, JNIEnv** newThreadEnv). I'm a little bit confused.Current thread hasn't been attached yet. So there is no JNIEnv for it yet. How it can be passed as the first parameter to vm_attach()? Then you will be always able to get the JavaVM from the JNIEnv. The only difference is that you are currently doing JNIEnv-JavaVM conversion in the VMThreadManager, but why can't you just do this in vm_attach() without changing the interface of the TM? So far JavaVM really looks like an extra knowledge that TM doesn't have to be aware of. Moreover there is no JNIEnv when main thread attaches to VM. So we should either keep it as is or change original design of TM and attach thread to VM before attaching it to TM. In that case we will have valid JNIEnv which can be passed to jthread_atatch. We need to think over it twice before changing something Right. For jthread_attach, JNIenv needs to be changed from being input parameter to being the output parameter. The way how JNIenv is obtained by TM should be vm_attach() event. OK, JNIEnv is output parameter. And it obtained by vm_attach(). This is exactly like I do in the patch. What I don't understand is how jthread_attach knows to which VM the thread should be attached? Do you suggest calling vm_attach first to create JNIEnv it pass it to jthread_attach? 4) Original classlib hythread do not use hythread_library_t in arguments, It uses following code: hythread_library_t lib = GLOBAL_DATA (default_library); or hythread_library_t library = thread-library; So could you please use the same mechanism and remove hythread_*_ex functions. Let's see if classlib's hythread needs changing first. It seems for me such code prevents us from having multi VM. 5. You introduce more then one jni env, but still use global variable for it. So all changes like following: -JNIEnv *jenv = (JNIEnv*)jni_native_intf; +JNIEnv *jenv = jni_native_intf; should be like: -JNIEnv *jenv = (JNIEnv*)jni_native_intf; +JNIEnv *jenv = get_jni_env(jthread_self()); Ok, I see. I agree that global jni_native_intf should not be used. There was simple
Re: [vote] HARMONY-1410 : JDWP agent for DRLVM
+1 -Stepan. On 9/28/06, Geir Magnusson Jr. wrote: BCC and ACQs are in. What say ye? Would it be nice to debug using eclipse debugger in DRLVM? [ ] + 1 accept this contribution into the project [ ] -1 don't accept (please give reason) Vote runs usual 3 days unless protest or early completion. geir -- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [drlvm] [launcher] Executable hangs
Alexey, Hmm, DRLVM build enforces static linking with APR artifacts built alongside the VM - it does not use any pre-installed APR. And I hardly think APR configure fails here... Nevertheless, Armand, maybe it worths trying a standalone APR test? Either simple dso_load scenario, or even APR unit tests (these are bundled with APR sources, just look for README). Sorry, but how can I run the standalone APR test? Where is the APR sources bundled? Thanks, Armand - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
