Re: Thoughts on possible options to JDK-8027598

[Rajan Halade]

On 11/27/2013 12:47, Bradford Wetmore wrote:

> jtreg team? I am not aware of one. Jon Gibbons works on this in his
> spare time.

I would call him "the JTREG team."

Was there an ETA for this?  Balancing recent test stabilization 
efforts with this, you may consider consider adding othervm and then 
back it out when the JTREG fix is made.
I am discussing this with Jon and will soon update with summary of it. I 
am also working on adding /othervm to affected tests.


- Rajan


Brad



On 11/25/2013 2:29 AM, Chris Hegarty wrote:

On 22/11/13 21:11, Rajan Halade wrote:


On 11/22/2013 11:42, Sean Mullan wrote:

On 11/21/2013 04:53 AM, Chris Hegarty wrote:
If I am correct, JTREG has support provider cleanup already. 
But the

question is even JTREG reset the providers, it still cannot ensure
next
test won't be impacted because in some circumstances JDK may 
need to

cache something which depends on previous providers. Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will
require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the 
security

group should ultimately have to agree/disagree, but we have similar
issues in other parts of the platform. I don't know the specifics of
the
code here so it may, or may not, be an issue.

If any part of the security framework stores values in static fields,
that is dependent on the security providers, then when the providers
change this static value may be incorrect.

We encounter this from time to time with system properties. E.g. JDK
HTTP Client code reads system property and stores in static field, 
JDK

HTTP Client will never never re-read the property as it believes it
will
not change. Having jtreg reset/clean certain system properties 
will not

help in this case.


Yes, the example with system properties is a good one.

However, providers are designed to be added and removed dynamically
(see the java.security.Security API), so if there is some static
information not being cleared when providers are reset, I would tend
to think it may be a bug in the JDK.


OK, if this is the case then that is fine.


yes. thanks!


My preference would be change jtreg to clear/restore providers, and
more thoroughly analyze any subsequent test failures as it may be a
bug in the JDK.
Ok, I will follow up with jtreg team to find out when they can 
commit to

timeline.


jtreg team? I am not aware of one. Jon Gibbons works on this in his
spare time.  You may want to take a look at jtreg [1] in the code tools
project.

-Chris.

[1] http://openjdk.java.net/projects/code-tools/jtreg/



- Rajan


--Sean

Re: Thoughts on possible options to JDK-8027598

[Bradford Wetmore]

> jtreg team? I am not aware of one. Jon Gibbons works on this in his
> spare time.

I would call him "the JTREG team."

Was there an ETA for this?  Balancing recent test stabilization efforts 
with this, you may consider consider adding othervm and then back it out 
when the JTREG fix is made.


Brad



On 11/25/2013 2:29 AM, Chris Hegarty wrote:

On 22/11/13 21:11, Rajan Halade wrote:


On 11/22/2013 11:42, Sean Mullan wrote:

On 11/21/2013 04:53 AM, Chris Hegarty wrote:

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure
next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers. Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will
require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the security
group should ultimately have to agree/disagree, but we have similar
issues in other parts of the platform. I don't know the specifics of
the
code here so it may, or may not, be an issue.

If any part of the security framework stores values in static fields,
that is dependent on the security providers, then when the providers
change this static value may be incorrect.

We encounter this from time to time with system properties. E.g. JDK
HTTP Client code reads system property and stores in static field, JDK
HTTP Client will never never re-read the property as it believes it
will
not change. Having jtreg reset/clean certain system properties will not
help in this case.


Yes, the example with system properties is a good one.

However, providers are designed to be added and removed dynamically
(see the java.security.Security API), so if there is some static
information not being cleared when providers are reset, I would tend
to think it may be a bug in the JDK.


OK, if this is the case then that is fine.


yes. thanks!


My preference would be change jtreg to clear/restore providers, and
more thoroughly analyze any subsequent test failures as it may be a
bug in the JDK.

Ok, I will follow up with jtreg team to find out when they can commit to
timeline.


jtreg team? I am not aware of one. Jon Gibbons works on this in his
spare time.  You may want to take a look at jtreg [1] in the code tools
project.

-Chris.

[1] http://openjdk.java.net/projects/code-tools/jtreg/



- Rajan


--Sean

Re: Thoughts on possible options to JDK-8027598

[Chris Hegarty]

On 22/11/13 21:11, Rajan Halade wrote:


On 11/22/2013 11:42, Sean Mullan wrote:

On 11/21/2013 04:53 AM, Chris Hegarty wrote:

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure
next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers. Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the security
group should ultimately have to agree/disagree, but we have similar
issues in other parts of the platform. I don't know the specifics of the
code here so it may, or may not, be an issue.

If any part of the security framework stores values in static fields,
that is dependent on the security providers, then when the providers
change this static value may be incorrect.

We encounter this from time to time with system properties. E.g. JDK
HTTP Client code reads system property and stores in static field, JDK
HTTP Client will never never re-read the property as it believes it will
not change. Having jtreg reset/clean certain system properties will not
help in this case.


Yes, the example with system properties is a good one.

However, providers are designed to be added and removed dynamically
(see the java.security.Security API), so if there is some static
information not being cleared when providers are reset, I would tend
to think it may be a bug in the JDK.


OK, if this is the case then that is fine.


yes. thanks!


My preference would be change jtreg to clear/restore providers, and
more thoroughly analyze any subsequent test failures as it may be a
bug in the JDK.

Ok, I will follow up with jtreg team to find out when they can commit to
timeline.


jtreg team? I am not aware of one. Jon Gibbons works on this in his 
spare time.  You may want to take a look at jtreg [1] in the code tools 
project.


-Chris.

[1] http://openjdk.java.net/projects/code-tools/jtreg/



- Rajan


--Sean

Re: Thoughts on possible options to JDK-8027598

[Rajan Halade]

On 11/22/2013 11:42, Sean Mullan wrote:

On 11/21/2013 04:53 AM, Chris Hegarty wrote:

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure 
next

test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers. Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the security
group should ultimately have to agree/disagree, but we have similar
issues in other parts of the platform. I don't know the specifics of the
code here so it may, or may not, be an issue.

If any part of the security framework stores values in static fields,
that is dependent on the security providers, then when the providers
change this static value may be incorrect.

We encounter this from time to time with system properties. E.g. JDK
HTTP Client code reads system property and stores in static field, JDK
HTTP Client will never never re-read the property as it believes it will
not change. Having jtreg reset/clean certain system properties will not
help in this case.


Yes, the example with system properties is a good one.

However, providers are designed to be added and removed dynamically 
(see the java.security.Security API), so if there is some static 
information not being cleared when providers are reset, I would tend 
to think it may be a bug in the JDK.

yes. thanks!


My preference would be change jtreg to clear/restore providers, and 
more thoroughly analyze any subsequent test failures as it may be a 
bug in the JDK.
Ok, I will follow up with jtreg team to find out when they can commit to 
timeline.


- Rajan


--Sean

Re: Thoughts on possible options to JDK-8027598

[Sean Mullan]

On 11/21/2013 04:53 AM, Chris Hegarty wrote:

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the security
group should ultimately have to agree/disagree, but we have similar
issues in other parts of the platform. I don't know the specifics of the
code here so it may, or may not, be an issue.

If any part of the security framework stores values in static fields,
that is dependent on the security providers, then when the providers
change this static value may be incorrect.

We encounter this from time to time with system properties. E.g. JDK
HTTP Client code reads system property and stores in static field, JDK
HTTP Client will never never re-read the property as it believes it will
not change. Having jtreg reset/clean certain system properties will not
help in this case.


Yes, the example with system properties is a good one.

However, providers are designed to be added and removed dynamically (see 
the java.security.Security API), so if there is some static information 
not being cleared when providers are reset, I would tend to think it may 
be a bug in the JDK.


My preference would be change jtreg to clear/restore providers, and more 
thoroughly analyze any subsequent test failures as it may be a bug in 
the JDK.


--Sean

Re: Thoughts on possible options to JDK-8027598

[Chris Hegarty]

On 11/20/2013 11:35 PM, Rajan Halade wrote:




Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to
store/restore security providers when run in agentvm mode.

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require
careful changes to the JRE itself.

Pardon my ignorance - if I gather correctly then ProvidersSnapshot
library also doesn't sandbox effects completely.


FYI. I am not part of the security team, and someone from the security 
group should ultimately have to agree/disagree, but we have similar 
issues in other parts of the platform. I don't know the specifics of the 
code here so it may, or may not, be an issue.


If any part of the security framework stores values in static fields, 
that is dependent on the security providers, then when the providers 
change this static value may be incorrect.


We encounter this from time to time with system properties. E.g. JDK 
HTTP Client code reads system property and stores in static field, JDK 
HTTP Client will never never re-read the property as it believes it will 
not change. Having jtreg reset/clean certain system properties will not 
help in this case.



If there are not too many of these tests, I don't think that
explicitly marking them othervm will hurt too much. And making them
reliable will reduce greatly the amount of time spent head scratching
chasing intermittent failures.

I agree with you, we need to consider using othervm mode. There are ~66
(1% of all reg tests) such tests which are run in agentvm, don't use
ProvidersSnapshot library, and are modifying providers. I think its safe
to go with othervm option, let me know.


IMHO, this is the right way to go here.

-Chris.



- Rajan


-Chris.



Xuelei


2. Update impacting tests to run in othervm - simple change but may
slow
down batch execution slightly.
3. Update each test to use library
java/security/KeyPairGenerator/Failover.java like done in
java/security/Provider tests - another easy change and tests would
continue to run agentvm but would have added overhead of restoring
providers.

We will continue to pursue option 1 but many not be possible. Option
2 &
3 above are equally good and are debatable so would like your thoughts
on it.

Thanks,
Rajan

Re: Thoughts on possible options to JDK-8027598

[Rajan Halade]

Thanks for your comments. See inline -

- Rajan

On 11/20/2013 03:54, Chris Hegarty wrote:

On 20/11/13 01:07, Xuelei Fan wrote:

On 11/20/2013 8:19 AM, Rajan Halade wrote:

I am working towards fixing JDK-8027598
, there are multiple
options available here and would appreciate your thoughts on this. It
was filed to address the issue at large reported inJDK-8027526
.

Problem - When a regression test is run in agentvm mode and alters
security providers, it can cause adverse effects on next tests executed
in the batch. We have been batteling with few intermittent failures
which are caused by scenario like this so I think it is important to
have this fix.

Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to
store/restore security providers when run in agentvm mode.

If I am correct, JTREG has support provider cleanup already. But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require 
careful changes to the JRE itself.
Pardon my ignorance - if I gather correctly then ProvidersSnapshot 
library also doesn't sandbox effects completely.


If there are not too many of these tests, I don't think that 
explicitly marking them othervm will hurt too much. And making them 
reliable will reduce greatly the amount of time spent head scratching 
chasing intermittent failures.
I agree with you, we need to consider using othervm mode. There are ~66 
(1% of all reg tests) such tests which are run in agentvm, don't use 
ProvidersSnapshot library, and are modifying providers. I think its safe 
to go with othervm option, let me know.


- Rajan


-Chris.



Xuelei

2. Update impacting tests to run in othervm - simple change but may 
slow

down batch execution slightly.
3. Update each test to use library
java/security/KeyPairGenerator/Failover.java like done in
java/security/Provider tests - another easy change and tests would
continue to run agentvm but would have added overhead of restoring
providers.

We will continue to pursue option 1 but many not be possible. Option 
2 &

3 above are equally good and are debatable so would like your thoughts
on it.

Thanks,
Rajan

Re: Thoughts on possible options to JDK-8027598

[Chris Hegarty]

On 20/11/13 01:07, Xuelei Fan wrote:

On 11/20/2013 8:19 AM, Rajan Halade wrote:

I am working towards fixing JDK-8027598
, there are multiple
options available here and would appreciate your thoughts on this. It
was filed to address the issue at large reported inJDK-8027526
.

Problem - When a regression test is run in agentvm mode and alters
security providers, it can cause adverse effects on next tests executed
in the batch. We have been batteling with few intermittent failures
which are caused by scenario like this so I think it is important to
have this fix.

Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to
store/restore security providers when run in agentvm mode.

If I am correct, JTREG has support provider cleanup already.  But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.


Right, any static or cached data may be invalid, and this will require 
careful changes to the JRE itself.


If there are not too many of these tests, I don't think that explicitly 
marking them othervm will hurt too much. And making them reliable will 
reduce greatly the amount of time spent head scratching chasing 
intermittent failures.


-Chris.



Xuelei


2. Update impacting tests to run in othervm - simple change but may slow
down batch execution slightly.
3. Update each test to use library
java/security/KeyPairGenerator/Failover.java like done in
java/security/Provider tests - another easy change and tests would
continue to run agentvm but would have added overhead of restoring
providers.

We will continue to pursue option 1 but many not be possible. Option 2 &
3 above are equally good and are debatable so would like your thoughts
on it.

Thanks,
Rajan

Re: Thoughts on possible options to JDK-8027598

[Bradford Wetmore]

On 11/19/2013 5:07 PM, Xuelei Fan wrote:

On 11/20/2013 8:19 AM, Rajan Halade wrote:

I am working towards fixing JDK-8027598
, there are multiple
options available here and would appreciate your thoughts on this. It
was filed to address the issue at large reported inJDK-8027526
.

Problem - When a regression test is run in agentvm mode and alters
security providers, it can cause adverse effects on next tests executed
in the batch. We have been batteling with few intermittent failures
which are caused by scenario like this so I think it is important to
have this fix.

Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to
store/restore security providers when run in agentvm mode.

If I am correct, JTREG has support provider cleanup already.


No, it does not currently.

Brad



But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.

Xuelei


2. Update impacting tests to run in othervm - simple change but may slow
down batch execution slightly.
3. Update each test to use library
java/security/KeyPairGenerator/Failover.java like done in
java/security/Provider tests - another easy change and tests would
continue to run agentvm but would have added overhead of restoring
providers.

We will continue to pursue option 1 but many not be possible. Option 2 &
3 above are equally good and are debatable so would like your thoughts
on it.

Thanks,
Rajan

Re: Thoughts on possible options to JDK-8027598

[Bradford Wetmore]

On 11/19/2013 4:19 PM, Rajan Halade wrote:

I am working towards fixing JDK-8027598
, there are multiple
options available here and would appreciate your thoughts on this. It
was filed to address the issue at large reported inJDK-8027526
.

Problem - When a regression test is run in agentvm mode and alters
security providers, it can cause adverse effects on next tests executed
in the batch. We have been batteling with few intermittent failures
which are caused by scenario like this so I think it is important to
have this fix.

Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to
store/restore security providers when run in agentvm mode.


That is my preference.  I'm also cc'ing Jonathan Gibbons who should also 
be involved.


I believe you can just look to see if the provider list has been 
updated, then restore only if that is the case.



2. Update impacting tests to run in othervm - simple change but may slow
down batch execution slightly.


Correct.

I am not suggesting the following, but one other thing that was proposed 
was to update the Makefiles to run the java_security* targets in 
othervm, but that doesn't work when calling jtreg directly, and impacts 
all tests in that target.



3. Update each test to use library
java/security/KeyPairGenerator/Failover.java like done in
java/security/Provider tests - another easy change and tests would
continue to run agentvm but would have added overhead of restoring
providers.

We will continue to pursue option 1 but many not be possible. Option 2 &
3 above are equally good and are debatable so would like your thoughts
on it.


I would suggest pursing in order 1, 3, 2.

Brad




Thanks,
Rajan

Re: Thoughts on possible options to JDK-8027598

[Xuelei Fan]

On 11/20/2013 8:19 AM, Rajan Halade wrote:
> I am working towards fixing JDK-8027598
> , there are multiple
> options available here and would appreciate your thoughts on this. It
> was filed to address the issue at large reported inJDK-8027526
> .
> 
> Problem - When a regression test is run in agentvm mode and alters
> security providers, it can cause adverse effects on next tests executed
> in the batch. We have been batteling with few intermittent failures
> which are caused by scenario like this so I think it is important to
> have this fix.
> 
> Possible approaches -
> 1. Enhance JTREG - This option would require change in jtreg to
> store/restore security providers when run in agentvm mode.
If I am correct, JTREG has support provider cleanup already.  But the
question is even JTREG reset the providers, it still cannot ensure next
test won't be impacted because in some circumstances JDK may need to
cache something which depends on previous providers.  Still need to
analysis the test case by case.

Xuelei

> 2. Update impacting tests to run in othervm - simple change but may slow
> down batch execution slightly.
> 3. Update each test to use library
> java/security/KeyPairGenerator/Failover.java like done in
> java/security/Provider tests - another easy change and tests would
> continue to run agentvm but would have added overhead of restoring
> providers.
> 
> We will continue to pursue option 1 but many not be possible. Option 2 &
> 3 above are equally good and are debatable so would like your thoughts
> on it.
> 
> Thanks,
> Rajan
>  
> 

Thoughts on possible options to JDK-8027598

[Rajan Halade]

I am working towards fixing JDK-8027598 
, there are multiple 
options available here and would appreciate your thoughts on this. It 
was filed to address the issue at large reported inJDK-8027526 
.


Problem - When a regression test is run in agentvm mode and alters 
security providers, it can cause adverse effects on next tests executed 
in the batch. We have been batteling with few intermittent failures 
which are caused by scenario like this so I think it is important to 
have this fix.


Possible approaches -
1. Enhance JTREG - This option would require change in jtreg to 
store/restore security providers when run in agentvm mode.
2. Update impacting tests to run in othervm - simple change but may slow 
down batch execution slightly.
3. Update each test to use library 
java/security/KeyPairGenerator/Failover.java like done in 
java/security/Provider tests - another easy change and tests would 
continue to run agentvm but would have added overhead of restoring 
providers.


We will continue to pursue option 1 but many not be possible. Option 2 & 
3 above are equally good and are debatable so would like your thoughts 
on it.


Thanks,
Rajan