[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259621#comment-14259621
]
ASF subversion and git services commented on LUCENE-6135:
-
Commit 1648189 from [~rcmuir] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1648189 ]
LUCENE-6135: renumber fields randomly in tests
> re-number fields randomly in tests
> --
>
> Key: LUCENE-6135
> URL: https://issues.apache.org/jira/browse/LUCENE-6135
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Robert Muir
> Attachments: LUCENE-6135.patch
>
>
> Currently some code (such as stored fields bulk merge) depends upon
> consistent field number ordering.
> In the case field numbers do not align, then optimizations are disabled,
> because the would cause crazy corruption where values are mixed up across
> different fields.
> But this is hardly tested today. If i introduce an intentional bug into this
> logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
> and only about 10% of the time at best. In general tests pass.
> {code}
> ---
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (revision 1647793)
> +++
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (working copy)
> @@ -52,6 +52,10 @@
> for (int i = 0; i < numReaders; i++) {
>for (FieldInfo fi : mergeState.fieldInfos[i]) {
> FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
> +// nocommit:
> +if (true) {
> + break;
> +}
> if (other == null || !other.name.equals(fi.name)) {
>continue nextReader;
> }
> {code}
> Don't get me wrong, its a great simple test, but it should not be the only
> one doing this. And it would be great if it failed more often, i havent
> looked as to why it only fails rarely if there is a bug in this stuff.
> But in general, we should simulate this more. My current idea is to shuffle
> up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259611#comment-14259611
]
ASF subversion and git services commented on LUCENE-6135:
-
Commit 1648188 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1648188 ]
LUCENE-6135: renumber fields randomly in tests
> re-number fields randomly in tests
> --
>
> Key: LUCENE-6135
> URL: https://issues.apache.org/jira/browse/LUCENE-6135
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Robert Muir
> Attachments: LUCENE-6135.patch
>
>
> Currently some code (such as stored fields bulk merge) depends upon
> consistent field number ordering.
> In the case field numbers do not align, then optimizations are disabled,
> because the would cause crazy corruption where values are mixed up across
> different fields.
> But this is hardly tested today. If i introduce an intentional bug into this
> logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
> and only about 10% of the time at best. In general tests pass.
> {code}
> ---
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (revision 1647793)
> +++
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (working copy)
> @@ -52,6 +52,10 @@
> for (int i = 0; i < numReaders; i++) {
>for (FieldInfo fi : mergeState.fieldInfos[i]) {
> FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
> +// nocommit:
> +if (true) {
> + break;
> +}
> if (other == null || !other.name.equals(fi.name)) {
>continue nextReader;
> }
> {code}
> Don't get me wrong, its a great simple test, but it should not be the only
> one doing this. And it would be great if it failed more often, i havent
> looked as to why it only fails rarely if there is a bug in this stuff.
> But in general, we should simulate this more. My current idea is to shuffle
> up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258342#comment-14258342
]
Robert Muir commented on LUCENE-6135:
-
+1 to dedicated tests in the stored fields format too!
The mockrandom MP idea is just a "shotgun" approach. It may help detect bugs
anywhere, not just stored fields.
But this should be a last resort, we should test these corner cases in the
stored fields base test heavily.
> re-number fields randomly in tests
> --
>
> Key: LUCENE-6135
> URL: https://issues.apache.org/jira/browse/LUCENE-6135
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Robert Muir
>
> Currently some code (such as stored fields bulk merge) depends upon
> consistent field number ordering.
> In the case field numbers do not align, then optimizations are disabled,
> because the would cause crazy corruption where values are mixed up across
> different fields.
> But this is hardly tested today. If i introduce an intentional bug into this
> logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
> and only about 10% of the time at best. In general tests pass.
> {code}
> ---
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (revision 1647793)
> +++
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (working copy)
> @@ -52,6 +52,10 @@
> for (int i = 0; i < numReaders; i++) {
>for (FieldInfo fi : mergeState.fieldInfos[i]) {
> FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
> +// nocommit:
> +if (true) {
> + break;
> +}
> if (other == null || !other.name.equals(fi.name)) {
>continue nextReader;
> }
> {code}
> Don't get me wrong, its a great simple test, but it should not be the only
> one doing this. And it would be great if it failed more often, i havent
> looked as to why it only fails rarely if there is a bug in this stuff.
> But in general, we should simulate this more. My current idea is to shuffle
> up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258340#comment-14258340
]
Adrien Grand commented on LUCENE-6135:
--
Maybe we should have a test decicated to this in the
BaseStoredFieldsFormatTestCase too (since this issue most likely hits stored
fields)?
> re-number fields randomly in tests
> --
>
> Key: LUCENE-6135
> URL: https://issues.apache.org/jira/browse/LUCENE-6135
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Robert Muir
>
> Currently some code (such as stored fields bulk merge) depends upon
> consistent field number ordering.
> In the case field numbers do not align, then optimizations are disabled,
> because the would cause crazy corruption where values are mixed up across
> different fields.
> But this is hardly tested today. If i introduce an intentional bug into this
> logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
> and only about 10% of the time at best. In general tests pass.
> {code}
> ---
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (revision 1647793)
> +++
> lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
> (working copy)
> @@ -52,6 +52,10 @@
> for (int i = 0; i < numReaders; i++) {
>for (FieldInfo fi : mergeState.fieldInfos[i]) {
> FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
> +// nocommit:
> +if (true) {
> + break;
> +}
> if (other == null || !other.name.equals(fi.name)) {
>continue nextReader;
> }
> {code}
> Don't get me wrong, its a great simple test, but it should not be the only
> one doing this. And it would be great if it failed more often, i havent
> looked as to why it only fails rarely if there is a bug in this stuff.
> But in general, we should simulate this more. My current idea is to shuffle
> up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
