[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469109#comment-16469109 ] ASF subversion and git services commented on LUCENE-8298: - Commit 0c65af048b3a497ea2e95e48c886b3a653412a0c in lucene-solr's branch refs/heads/branch_7x from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0c65af0 ] LUCENE-8298: Allow DocValues updates to reset a value Today once a document has a value in a certain DV field this values can only be changed but not removed. While resetting / removing a value from a field is certainly a corner case it can be used to undelete a soft-deleted document unless it's merged away. This allows to rollback changes without rolling back to another commitpoint or trashing all uncommitted changes. In certain cenarios it can be used to "repair" history of documents in distributed systems. > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch, > LUCENE-8298.patch, LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469090#comment-16469090 ] ASF subversion and git services commented on LUCENE-8298: - Commit a3c86373e47ddd43368c5800eea63b50d2064759 in lucene-solr's branch refs/heads/master from [~simonw] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a3c8637 ] LUCENE-8298: Allow DocValues updates to reset a value Today once a document has a value in a certain DV field this values can only be changed but not removed. While resetting / removing a value from a field is certainly a corner case it can be used to undelete a soft-deleted document unless it's merged away. This allows to rollback changes without rolling back to another commitpoint or trashing all uncommitted changes. In certain cenarios it can be used to "repair" history of documents in distributed systems. > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch, > LUCENE-8298.patch, LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469033#comment-16469033 ] Adrien Grand commented on LUCENE-8298: -- +1 > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch, > LUCENE-8298.patch, LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[
https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469027#comment-16469027
]
Simon Willnauer commented on LUCENE-8298:
-
> I'd rather like not to add {{Bits#getMutableCopy}} and keep the {{Bits}} API
> minimal. Otherwise +1.
fair enough. I agree lets keep it clean. I used a static method on FixedBitset
instead.
> Allow DocValues updates to reset a value
>
>
> Key: LUCENE-8298
> URL: https://issues.apache.org/jira/browse/LUCENE-8298
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch,
> LUCENE-8298.patch, LUCENE-8298.patch
>
>
> Today once a document has a value in a certain DV field this values
> can only be changed but not removed. While resetting / removing a value
> from a field is certainly a corner case it can be used to undelete a
> soft-deleted document unless it's merged away.
> This allows to rollback changes without rolling back to another
> commitpoint
> or trashing all uncommitted changes. In certain cenarios it can be used to
> "repair" history of documents in distributed systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[
https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468936#comment-16468936
]
Adrien Grand commented on LUCENE-8298:
--
I'd rather like not to add {{Bits#getMutableCopy}} and keep the {{Bits}} API
minimal. Otherwise +1.
> Allow DocValues updates to reset a value
>
>
> Key: LUCENE-8298
> URL: https://issues.apache.org/jira/browse/LUCENE-8298
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch,
> LUCENE-8298.patch
>
>
> Today once a document has a value in a certain DV field this values
> can only be changed but not removed. While resetting / removing a value
> from a field is certainly a corner case it can be used to undelete a
> soft-deleted document unless it's merged away.
> This allows to rollback changes without rolling back to another
> commitpoint
> or trashing all uncommitted changes. In certain cenarios it can be used to
> "repair" history of documents in distributed systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468879#comment-16468879 ] Simon Willnauer commented on LUCENE-8298: - [~jpountz] I integrated with your latest changes > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch, > LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468781#comment-16468781 ] Simon Willnauer commented on LUCENE-8298: - I updated the patch [~jpountz] > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch, LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[
https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468560#comment-16468560
]
Adrien Grand commented on LUCENE-8298:
--
I think we should try to avoid adding {{set}} on MutableBits. It puts too much
pressure on the live docs format? Also the ability to un-hard-delete is a bit
scary to me because of the documents that got marked as deleted because they
failed indexing. Maybe {{PendingSoftDeletes}} should manage their bits directly
instead of relying on the live docs format?
Also do we need to detect null values more proactively now that the
NumericDocValuesField constructor allows to pass nulls?
> Allow DocValues updates to reset a value
>
>
> Key: LUCENE-8298
> URL: https://issues.apache.org/jira/browse/LUCENE-8298
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8298.patch, LUCENE-8298.patch
>
>
> Today once a document has a value in a certain DV field this values
> can only be changed but not removed. While resetting / removing a value
> from a field is certainly a corner case it can be used to undelete a
> soft-deleted document unless it's merged away.
> This allows to rollback changes without rolling back to another
> commitpoint
> or trashing all uncommitted changes. In certain cenarios it can be used to
> "repair" history of documents in distributed systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[
https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467604#comment-16467604
]
Michael McCandless commented on LUCENE-8298:
+1, very cool! My favorite part of the patch is:
{quote}just don't do it!{quote}
:)
Curiously, Lucene long ago had the ability to undelete a document! It was a
method on {{IndexReader}} to undelete all documents (that had not been
reclaimed by merges yet), and now we get that ability back, but in the right
place ({{IndexWriter}}) and with control over which doc to un-soft-delete.
> Allow DocValues updates to reset a value
>
>
> Key: LUCENE-8298
> URL: https://issues.apache.org/jira/browse/LUCENE-8298
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8298.patch, LUCENE-8298.patch
>
>
> Today once a document has a value in a certain DV field this values
> can only be changed but not removed. While resetting / removing a value
> from a field is certainly a corner case it can be used to undelete a
> soft-deleted document unless it's merged away.
> This allows to rollback changes without rolling back to another
> commitpoint
> or trashing all uncommitted changes. In certain cenarios it can be used to
> "repair" history of documents in distributed systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467166#comment-16467166 ] Simon Willnauer commented on LUCENE-8298: - new patch with added javadocs, API cleanups and more tests > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch, LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8298) Allow DocValues updates to reset a value
[ https://issues.apache.org/jira/browse/LUCENE-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465964#comment-16465964 ] Simon Willnauer commented on LUCENE-8298: - I attached a patch for discussion. I need to do some cleanups, more tests and clarify javadocs but it shows the idea > Allow DocValues updates to reset a value > > > Key: LUCENE-8298 > URL: https://issues.apache.org/jira/browse/LUCENE-8298 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8298.patch > > > Today once a document has a value in a certain DV field this values > can only be changed but not removed. While resetting / removing a value > from a field is certainly a corner case it can be used to undelete a > soft-deleted document unless it's merged away. > This allows to rollback changes without rolling back to another > commitpoint > or trashing all uncommitted changes. In certain cenarios it can be used to > "repair" history of documents in distributed systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
