[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786229#comment-15786229 ] ASF subversion and git services commented on LUCENE-7595: - Commit 40a8b4edb4cfc7de5b62037fdcb389afa247573d in lucene-solr's branch refs/heads/branch_6x from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=40a8b4e ] LUCENE-7595: Disable another test not compatible with RamUsageTester > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Fix For: 6.x, master (7.0), 6.4 > > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786225#comment-15786225 ] ASF subversion and git services commented on LUCENE-7595: - Commit d65c02e8cc14f03389c2426ea3d3ddd75e12b1ec in lucene-solr's branch refs/heads/master from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d65c02e ] LUCENE-7595: Disable another test not compatible with RamUsageTester > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Fix For: 6.x, master (7.0), 6.4 > > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785955#comment-15785955 ] ASF subversion and git services commented on LUCENE-7595: - Commit 80512ec412c20517341ddd50c78baf5270fcdc2f in lucene-solr's branch refs/heads/branch_6x from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=80512ec ] LUCENE-7595: Fix bug with RamUsageTester incorrectly handling Iterables outside Java Runtime > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Fix For: 6.x, master (7.0), 6.4 > > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785952#comment-15785952 ] ASF subversion and git services commented on LUCENE-7595: - Commit db9190db9372ae88a7392a7186397441ce070a96 in lucene-solr's branch refs/heads/master from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=db9190d ] LUCENE-7595: Fix bug with RamUsageTester incorrectly handling Iterables outside Java Runtime > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Fix For: 6.x, master (7.0), 6.4 > > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784166#comment-15784166 ] ASF subversion and git services commented on LUCENE-7595: - Commit ea7db0a176979559e874b292522fa7006b578882 in lucene-solr's branch refs/heads/branch_6x from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ea7db0a ] LUCENE-7595: Improve RAMUsageTester in test-framework to estimate memory usage of runtime classes and work with Java 9 EA (b148+). Disable static field heap usage checker in LuceneTestCase > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784137#comment-15784137 ] ASF subversion and git services commented on LUCENE-7595: - Commit f29d2b5668296dfcdb8d650305449674faa29847 in lucene-solr's branch refs/heads/master from [~thetaphi] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f29d2b5 ] LUCENE-7595: Improve RAMUsageTester in test-framework to estimate memory usage of runtime classes and work with Java 9 EA (b148+). Disable static field heap usage checker in LuceneTestCase > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > Attachments: LUCENE-7595.patch > > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[
https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15779982#comment-15779982
]
Dawid Weiss commented on LUCENE-7595:
-
Looks good to me overall. A few notes and ideas.
There is a separate RUE impl. inside randomized runner just for the sake of
{{StaticFieldsInvariantRule}}... perhaps we could simply copy over this rule's
source code to Lucene to make use of the local ram estimator -- then you'd have
full control to tweak it as needed. Perhaps it'd also simplify the static rule
initialization block.
{code}
+ a(String.class, v -> charArraySize(v.length())); // may not be correct
with Java 9's compact strings!
{code}
Yup, this is overestimating based on the patch
(https://bugs.openjdk.java.net/browse/JDK-8054307). But I think it's ok --
better to overestimate here and blow up early and there's no way to check which
representation (byte or char-based) was actually chosen by the jvm. Yes, we
could reimplement the Latin1 vs. UTF16 heuristic, but it seems like an
overkill. :)
> RAMUsageTester in test-framework and static field checker no longer works
> with Java 9
> -
>
> Key: LUCENE-7595
> URL: https://issues.apache.org/jira/browse/LUCENE-7595
> Project: Lucene - Core
> Issue Type: Bug
> Components: general/test
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Labels: Java9
> Attachments: LUCENE-7595.patch
>
>
> Lucene/Solr tests have a special rule that records memory usage in static
> fields before and after test, so we can detect memory leaks. This check dives
> into JDK classes (like java.lang.String to detect their size). As Java 9
> build 148 completely forbids setAccessible on any runtime class, we have to
> change or disable this check:
> - As first step I will only add the rule to LTC, if we not have Java 8
> - As a second step we might investigate how to improve this
> [~rcmuir] had some ideas for the 2nd point:
> - Don't dive into classes from JDK modules and instead "estimate" the size
> for some special cases (like Strings)
> - Disallow any static field in tests that is not final (constant) and points
> to an Object except: Strings and native (wrapper) types.
> In addition we also have RAMUsageTester, that has similar problems and is
> used to compare estimations of Lucene's calculations of
> Codec/IndexWriter/IndexReader memory usage with reality. We should simply
> disable those tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9
[ https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15753834#comment-15753834 ] Dawid Weiss commented on LUCENE-7595: - I agree certain classes could be approximated (like String, Lists, etc.). bq. Disallow any static field in tests that is not final (constant) and points to an Object except: Strings and native (wrapper) types. The check could be less strict -- we could fail if the value of such a field is non-null after the test and permit nullified reference fields. > RAMUsageTester in test-framework and static field checker no longer works > with Java 9 > - > > Key: LUCENE-7595 > URL: https://issues.apache.org/jira/browse/LUCENE-7595 > Project: Lucene - Core > Issue Type: Bug > Components: general/test >Reporter: Uwe Schindler >Assignee: Uwe Schindler > Labels: Java9 > > Lucene/Solr tests have a special rule that records memory usage in static > fields before and after test, so we can detect memory leaks. This check dives > into JDK classes (like java.lang.String to detect their size). As Java 9 > build 148 completely forbids setAccessible on any runtime class, we have to > change or disable this check: > - As first step I will only add the rule to LTC, if we not have Java 8 > - As a second step we might investigate how to improve this > [~rcmuir] had some ideas for the 2nd point: > - Don't dive into classes from JDK modules and instead "estimate" the size > for some special cases (like Strings) > - Disallow any static field in tests that is not final (constant) and points > to an Object except: Strings and native (wrapper) types. > In addition we also have RAMUsageTester, that has similar problems and is > used to compare estimations of Lucene's calculations of > Codec/IndexWriter/IndexReader memory usage with reality. We should simply > disable those tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
