[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756430#comment-13756430 ] Shai Erera commented on LUCENE-5178: I manage to reproduce it only if I set both tests.seed and tests.postingsformat. What I see is that Lucene42DVProducer is used, which returns MatchAllBits (it supports only SortedSet?), which leads to the incorrect counts. The initialized Codec is Lucene45Codec, which returns Lucene42DVF. The test does use defaultCodecSupportsDocsWithField(), but the latter only asserts that the name of the Codec is not 40, 41, 42, yet it does not check if the DVF supports it. That seems wrong? doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Fix For: 5.0, 4.5 Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756523#comment-13756523 ] Michael McCandless commented on LUCENE-5178: It sounds like we need to check the actual DVFormat for that field (_TestUtil.getDocValuesFormat(field)) and then test whether that format supports missing values. I think this failure can only happen if you explicitly set -Dtests.postingsformat, because then we make an anon subclass of Lucene45 (TestRuleSetupAndRestoreClassEnv.java at line 194) ... so it sounds like in general we should not be using defaultCodecSupportsDocsWithField() but rather something like defaultDVFormatSupportsDocsWithField(String field) ... doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Fix For: 5.0, 4.5 Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756536#comment-13756536 ] Shai Erera commented on LUCENE-5178: I see. I think this can also happen if you use RandomCodec and it draws Lucene42DVF? So in this case, with this seed, it trips if you set postingsformat, but I'm not sure that in general this assume() is correct. The ugly part of having a test calling _TestUtil.geDVF(field) (or we wrap it in a nice method) is that the test will need to decide up front on all the fields it uses, and if there's a mistake, the error may happen in the future and harder to debug (i.e. spot that the test uses a different field than what it passed to assume()). But I don't think that asserting the Codec is the right test here, so this has to change. doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Fix For: 5.0, 4.5 Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756551#comment-13756551 ] Shai Erera commented on LUCENE-5178: Opened LUCENE-5199. doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Fix For: 5.0, 4.5 Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756687#comment-13756687 ] Robert Muir commented on LUCENE-5178: - Can this commit please be reverted? The change makes the test API so complicated for something that cannot happen: You cannot have unsupported fields its all or none. This is a bug in LuceneTestCase, it should not do this when someone uses -Dtests.postingsformat. doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Fix For: 5.0, 4.5 Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[
https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13756313#comment-13756313
]
Han Jiang commented on LUCENE-5178:
---
During test I somehow hit a failure:
{noformat}
[junit4] FAILURE 0.27s | TestRangeAccumulator.testMissingValues
[junit4] Throwable #1: org.junit.ComparisonFailure: expected:...(0)
[junit4] less than 10 ([8)
[junit4] less than or equal to 10 (]8)
[junit4] over 90 (8)
[junit4] 9... but was:...(0)
[junit4] less than 10 ([28)
[junit4] less than or equal to 10 (2]8)
[junit4] over 90 (8)
[junit4] 9...
[junit4]at
__randomizedtesting.SeedInfo.seed([815B6AA86D05329C:EBC638EE498F066D]:0)
[junit4]at
org.apache.lucene.facet.range.TestRangeAccumulator.testMissingValues(TestRangeAccumulator.java:670)
[junit4]at java.lang.Thread.run(Thread.java:722)
{noformat}
Seed:
{noformat}
ant test -Dtestcase=TestRangeAccumulator -Dtests.method=testMissingValues
-Dtests.seed=815B6AA86D05329C -Dtests.slow=true -Dtests.postingsformat=Lucene41
-Dtests.locale=ca -Dtests.timezone=Australia/Currie -Dtests.file.encoding=UTF-8
{noformat}
doc values should expose missing values (or allow configurable defaults)
Key: LUCENE-5178
URL: https://issues.apache.org/jira/browse/LUCENE-5178
Project: Lucene - Core
Issue Type: Improvement
Reporter: Yonik Seeley
Fix For: 5.0, 4.5
Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch
DocValues should somehow allow a configurable default per-field.
Possible implementations include setting it on the field in the document or
registration of an IndexWriter callback.
If we don't make the default configurable, then another option is to have
DocValues fields keep track of whether a value was indexed for that document
or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13744870#comment-13744870 ] Michael McCandless commented on LUCENE-5178: +1, I reviewed the first patch. We'll need to fix facet module's dynamic range faceting to skip missing values; I can do this after you commit this patch... doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745467#comment-13745467 ] ASF subversion and git services commented on LUCENE-5178: - Commit 1515977 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1515977 ] LUCENE-5178: add missing support for docvalues doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745469#comment-13745469 ] Robert Muir commented on LUCENE-5178: - Thanks Mike: I added additional tests and ensured the 'missing' stuff in solr is fully functional. I'll let jenkins chomp on it for a while in trunk. doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745561#comment-13745561 ] ASF subversion and git services commented on LUCENE-5178: - Commit 1515999 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1515999 ] LUCENE-5178: handle missing values in range facets doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745571#comment-13745571 ] ASF subversion and git services commented on LUCENE-5178: - Commit 1516003 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1516003 ] LUCENE-5178: test requires codec that supports docsWithField doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745574#comment-13745574 ] ASF subversion and git services commented on LUCENE-5178: - Commit 1516012 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1516012 ] LUCENE-5178: add missing support for docvalues doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley Attachments: LUCENE-5178.patch, LUCENE-5178_reintegrate.patch DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13742182#comment-13742182 ] Yonik Seeley commented on LUCENE-5178: -- Yes, I think tracking/exposing missing values is the best option, esp for numerics where you can use the full range and still tell of there was a value or not. doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should expose missing values (or allow configurable defaults)
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13742186#comment-13742186 ] Robert Muir commented on LUCENE-5178: - OK. I can remove the solr defaultValue check here too: i have to fix the tests to test sort missing first/last / facet missing etc anyway (currently the dv tests avoid that). doc values should expose missing values (or allow configurable defaults) Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
