[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500154#comment-16500154 ] Nhat Nguyen commented on LUCENE-8335: - Thanks everyone! > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499812#comment-16499812 ] ASF subversion and git services commented on LUCENE-8335: - Commit e7a0a12926c399758a4021715a7419e22e59dab6 in lucene-solr's branch refs/heads/master from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e7a0a12 ] LUCENE-8335: Enforce soft-deletes field up-front Soft deletes field must be marked as such once it's introduced and can't be changed after the fact. Co-authored-by: Nhat Nguyen > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499811#comment-16499811 ] ASF subversion and git services commented on LUCENE-8335: - Commit 67b6593e7adbce76532e285cef42118e6cc3448f in lucene-solr's branch refs/heads/branch_7x from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=67b6593 ] LUCENE-8335: Enforce soft-deletes field up-front Soft deletes field must be marked as such once it's introduced and can't be changed after the fact. Co-authored-by: Nhat Nguyen > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499529#comment-16499529
]
Lucene/Solr QA commented on LUCENE-8335:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
34s{color} | {color:green} codecs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m
36s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} highlighter in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
34s{color} | {color:green} memory in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
27s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 71m 41s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.TestDistributedSearch |
| | solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8335 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926255/LUCENE-8335.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8
21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 3dc4fa1 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-LUCENE-Build/25/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/25/testReport/ |
| modules | C: lucene/codecs lucene/core lucene/highlighter lucene/memory
lucene/test-framework solr/core U: . |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/25/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Do not allow changing soft-deletes field
>
>
> Key: LUCENE-8335
> URL: https://issues.apache.org/jira/browse/LUCENE-8335
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Assignee: Simon Willnauer
>Priority: Minor
> Attachments: LUCENE-8335.patch, LUCENE-8335.patch, LUCENE-8335.patch
>
>
> Today we do not enforce an index to use a single soft-deletes field. A user
> can create an index with one soft-deletes field then open an IW with another
> field or add an index with a different soft-deletes field. This should not be
> allowed and reported the error to users as soon as possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499210#comment-16499210 ] Nhat Nguyen commented on LUCENE-8335: - [~simonw] I submitted a new patch. Can you please have a look when you have a few minutes. Thank you! > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499198#comment-16499198
]
Lucene/Solr QA commented on LUCENE-8335:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 2m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
12s{color} | {color:green} codecs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 47m
1s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
11s{color} | {color:green} highlighter in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
18s{color} | {color:green} memory in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
9s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}121m 27s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}215m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.api.collections.ShardSplitTest |
| | solr.cloud.PeerSyncReplicationTest |
| | solr.cloud.MoveReplicaHDFSTest |
| | solr.cloud.autoscaling.sim.TestTriggerIntegration |
| | solr.cloud.autoscaling.sim.TestLargeCluster |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8335 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926109/LUCENE-8335.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 3dc4fa1 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 8 2015 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-LUCENE-Build/23/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/23/testReport/ |
| modules | C: lucene/codecs lucene/core lucene/highlighter lucene/memory
lucene/test-framework solr/core U: . |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/23/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Do not allow changing soft-deletes field
>
>
> Key: LUCENE-8335
> URL: https://issues.apache.org/jira/browse/LUCENE-8335
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Assignee: Simon Willnauer
>Priority: Minor
> Attachments: LUCENE-8335.patch, LUCENE-8335.patch
>
>
> Today we do not enforce an index to use a single soft-deletes field. A user
> can create an index with one soft-deletes field then open an IW with another
> field or add an index with a different soft-deletes field. This should not be
> allowed and reported the error to users as soon as possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands,
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499035#comment-16499035 ] Nhat Nguyen commented on LUCENE-8335: - [~simonw] I am looking at this now. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499021#comment-16499021 ] ASF subversion and git services commented on LUCENE-8335: - Commit 3dc4fa199c175ed6351f66bac5c23c73b4e3f89a in lucene-solr's branch refs/heads/master from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3dc4fa1 ] Revert "LUCENE-8335: Enforce soft-deletes field up-front." This reverts commit a2d927667418d17a1f5f31a193092d5b04a4219e. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499019#comment-16499019
]
Simon Willnauer commented on LUCENE-8335:
-
[~dnhatn] I had an error porting this to branch_7x
{noformat}
[junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexingSequenceNumbers -Dtests.method=testStressUpdateSameID
-Dtests.seed=8A6CEEA00F01526D -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=fr-CH -Dtests.timezone=America/Santarem -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 2.73s J1 |
TestIndexingSequenceNumbers.testStressUpdateSameID <<<
[junit4]> Throwable #1: java.lang.IllegalArgumentException: this index
has [___soft_deletes] as soft-deletes already but soft-deletes field is not
configured in IWC
[junit4]>at
__randomizedtesting.SeedInfo.seed([8A6CEEA00F01526D:9BE73AE3C68F28BA]:0)
[junit4]>at
org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:283)
[junit4]>at
org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:428)
[junit4]>at
org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:459)
[junit4]>at
org.apache.lucene.index.ParallelLeafReader.(ParallelLeafReader.java:132)
[junit4]>at
org.apache.lucene.index.ParallelCompositeReader$1.(ParallelCompositeReader.java:126)
[junit4]>at
org.apache.lucene.index.ParallelCompositeReader.prepareLeafReaders(ParallelCompositeReader.java:126)
[junit4]>at
org.apache.lucene.index.ParallelCompositeReader.(ParallelCompositeReader.java:72)
[junit4]>at
org.apache.lucene.index.ParallelCompositeReader.(ParallelCompositeReader.java:65)
[junit4]>at
org.apache.lucene.index.ParallelCompositeReader.(ParallelCompositeReader.java:59)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.wrapReader(LuceneTestCase.java:1674)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.maybeWrapReader(LuceneTestCase.java:1739)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.newSearcher(LuceneTestCase.java:1864)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.newSearcher(LuceneTestCase.java:1849)
[junit4]>at
org.apache.lucene.util.LuceneTestCase.newSearcher(LuceneTestCase.java:1841)
[junit4]>at
org.apache.lucene.index.TestIndexingSequenceNumbers.testStressUpdateSameID(TestIndexingSequenceNumbers.java:127)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Lucene70,
sim=RandomSimilarity(queryNorm=false): {id=DFR I(n)2}, locale=fr-CH,
timezone=America/Santarem
[junit4] 2> NOTE: Mac OS X 10.13.4 x86_64/Oracle Corporation 1.8.0_144
(64-bit)/cpus=4,threads=1,free=195407280,total=334495744
[junit4] 2> NOTE: All tests run in this JVM: [TestBooleanScorer,
TestParallelCompositeReader, TestForTooMuchCloning, TestCheckIndex,
TestFilterIterator, TestTermScorer, TestFutureObjects,
TestAllFilesHaveChecksumFooter, TestSoftDeletesDirectoryReaderWrapper,
TestUniqueTermCount, TestFixedBitSet, TestIndexWriterOutOfFileDescriptors,
TestPointQueries, TestCrashCausesCorruptIndex,
TestGraphTokenStreamFiniteStrings, TestSortedSetSelector,
TestIndexWriterMergePolicy, TestByteSlices, TestCustomNorms,
TestExceedMaxTermLength, TestMultiThreadTermVectors, TestLongValuesSource,
TestByteArrayDataInput, TestForceMergeForever, TestSegmentReader,
TestMixedDocValuesUpdates, TestSparseFixedBitSet, TestSortedSetDocValues,
TestSpanBoostQuery, TestReqExclBulkScorer, TestOmitTf, TestPendingDeletes,
TestPendingSoftDeletes, Test4GBStoredFields, TestSimpleAttributeImpl,
TestAtomicUpdate, TestRAMDirectory, TestIndexedDISI, TestTimSorterWorstCase,
TestPriorityQueue, TestLiveFieldValues, TestSpanExplanations,
TestOmitPositions, TestIndexWriterDelete, TestTotalHitCountCollector,
TestMatchNoDocsQuery, TestDocValuesScoring, TestForUtil, TestNIOFSDirectory,
TestDemo, TestLucene70NormsFormat, TestCloseableThreadLocal,
TestDoubleRangeFieldQueries, LimitedFiniteStringsIteratorTest,
TestLucene70SegmentInfoFormat, TestIndexWriterOnDiskFull,
TestBufferedIndexInput, TestTermRangeQuery, TestMultiTermConstantScore,
TestPrefixCodedTerms, TestOfflineSorter, TestSortedNumericSortField,
TestIndexWriterDeleteByQuery, TestSpans, TestFlushByRamOrCountsPolicy,
TestFeatureField, TestLucene50TermVectorsFormat, TestLongPostings,
TestFuzzyQuery, TestTermsEnum2, TestMergeRateLimiter, TestGraphTokenizers,
TestCustomTermFreq, TestCharsRef, TestConcurrentMergeScheduler, TestDocValues,
TestMaxPosition, TestWildcard, TestOperations, TestIndexingSequenceNumbers]
{noformat}
can you take another look. I will revert the change to master for now.
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499009#comment-16499009 ] ASF subversion and git services commented on LUCENE-8335: - Commit a2d927667418d17a1f5f31a193092d5b04a4219e in lucene-solr's branch refs/heads/master from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a2d9276 ] LUCENE-8335: Enforce soft-deletes field up-front. Soft deletes field must be marked as such once it's introduced and can't be changed after the fact. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498978#comment-16498978 ] Simon Willnauer commented on LUCENE-8335: - [~dnhatn] this looks great I will push this in a bit. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16498147#comment-16498147 ] Nhat Nguyen commented on LUCENE-8335: - [~simonw] Thanks for reviewing. I've updated the new patch which addresses your comments. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch, LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497723#comment-16497723 ] Simon Willnauer commented on LUCENE-8335: - [~dnhatn] I think your patch looks good, here are a couple of comments. * can we make the hashmap a string and check if it's equal or null. I don't think we should have anything mutable here and we won't allow more than one field, right? * should we also fail if you open a writer that has no soft-deletes field configured but there is a field in the field infos that has the flag set? * I also think we should fail if the field already exists and is not a soft_deletes field but then get's marked as such. Like you can only have a soft_deletes field if you mark it as such from the very beginning ie. as soon as you introduce it. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497261#comment-16497261 ] Robert Muir commented on LUCENE-8335: - as a followup, I think lets support checkindex validation (e.g. LUCENE-8341). I am happy to see LUCENE-8342 opened up, thats probably an easier win than int vs float anyway, but it addresses the kind of concerns i had here, I think its important to also enforce stuff for typical usecases as well. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495109#comment-16495109 ] Robert Muir commented on LUCENE-8335: - Mike, I think nearly all of your same arguments could be made for preventing the change of an IntPoint field to a FloatPoint. But as it is now, any "schema" stuff in lucene is so minimum that, it doesn't know the difference. In the past the reasoning has been to keep it minimal and leave that to the consuming app. I don't think it makes sense that it can only track expert usecases. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493646#comment-16493646
]
Michael McCandless commented on LUCENE-8335:
[~rcmuir] do you have a use case in mind where a user would actually need to
change their soft deletes field? I.e. are you worried that we are preventing
such use cases? I can't think of one.
I think adding this protection (catching users who think they can safely change
their soft deletes field) is a natural continuation of steps we've already
taken to make it first class feature, e.g. adding
{{IndexWriterConfig.softDeletesField}}?
And it's dangerous now if a user writes to an index with different soft deletes
fields in different sessions, where the second session won't detect soft
deletes done in the first sessions. E.g. you can't do the same thing with hard
deletes – there is only one set of hard deletes and you can't change that from
one IW session to another.
{quote}[~mikemccand] indicated he want's to use it
{quote}
+1 – we want to use soft deletes to track recently deleted documents that we
can't really delete until the "out of order updates window" closes. Today we
are doing this by indexing explicit tombstone documents, and I suspect soft
deletes will be cleaner.
> Do not allow changing soft-deletes field
>
>
> Key: LUCENE-8335
> URL: https://issues.apache.org/jira/browse/LUCENE-8335
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Assignee: Simon Willnauer
>Priority: Minor
> Attachments: LUCENE-8335.patch
>
>
> Today we do not enforce an index to use a single soft-deletes field. A user
> can create an index with one soft-deletes field then open an IW with another
> field or add an index with a different soft-deletes field. This should not be
> allowed and reported the error to users as soon as possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492483#comment-16492483 ] Simon Willnauer commented on LUCENE-8335: - > i dont think lucene needs to enforce this. from my perspective its just a >docvalues field. given that lucene doesnt even know the difference between a >integer amd a rloat field, i dont think it should be tracking expert shit for >elasticsearch. my reasoning here is that in-turn to make this a more widely useable feature ie. [~mikemccand] indicated he want's to use it and I suspect the usecase is becoming more widely adopted we can be more strict about it and make it a non-expert feature. The fact that it's just a DV field is great and under the hood not many changes were necessary. The overhead of tracking this is small in my opinion and enforcing this would allow us to make the feature much less trappy down the road. ie. factory methods can automatically wrap indices that have a soft-deletes field, we can track the numSoftDeletes which helps a ton with assertions and allows to pull index stats by reading seginfos alone without opening a reader. I can work around all these thing and I am not even convinced we should do automatically wrapping a reader etc. but we can make this feature easy to use with a simple setter on IWC. From a interface perspective it's simple and the changes necessary to make it way less trappy warrent the change. I wonder what others think. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492192#comment-16492192
]
Lucene/Solr QA commented on LUCENE-8335:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
20s{color} | {color:green} codecs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 26m
26s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
3s{color} | {color:green} highlighter in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} memory in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
9s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 43s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster |
| | solr.cloud.TestAuthenticationFramework |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8335 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925273/LUCENE-8335.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8
21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / e5998fc |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-LUCENE-Build/18/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/18/testReport/ |
| modules | C: lucene/codecs lucene/core lucene/highlighter lucene/memory
lucene/test-framework solr/core U: . |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/18/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Do not allow changing soft-deletes field
>
>
> Key: LUCENE-8335
> URL: https://issues.apache.org/jira/browse/LUCENE-8335
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Assignee: Simon Willnauer
>Priority: Minor
> Attachments: LUCENE-8335.patch
>
>
> Today we do not enforce an index to use a single soft-deletes field. A user
> can create an index with one soft-deletes field then open an IW with another
> field or add an index with a different soft-deletes field. This should not be
> allowed and reported the error to users as soon as possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491970#comment-16491970 ] Robert Muir commented on LUCENE-8335: - i dont think lucene needs to enforce this. from my perspective its just a docvalues field. given that lucene doesnt even know the difference between a integer amd a rloat field, i dont think it should be tracking expert shit for elasticsearch. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
