[
https://issues.apache.org/jira/browse/USERGRID-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236149#comment-14236149
]
Todd Nine commented on USERGRID-255:
------------------------------------
This looks good. It should fit within our DataMigration implementation as
well. The DataMigration makes no assumption about the work it performs, so
this should work. I think the tool should combine these steps, but with the
following flow and existing code.
This would take place in the migrate call in the DataMigration impl
# Use the EntityIndex.addIndex to add the aliases and the new write index
# Use the existing index-rebuild functionality. (maybe refactor it into it's
own observable flow so it can be done in parallel batches for faster execution)
# Use new functionality EntityIndex.removePreviousIndexes(). This would
perform the following.
## Get the current write index from the write alias
## Get the current read indexes from the read alias
## Subtract the write index from the set of read indexes since it is the most
current and should be retained
## Drop remaining indexes in the set
Note that invoking this operation means potential data loss in the case that an
index rebuild is not run before removePreviousIndexes is invoked.
> Re-indexer That Removes Source from ES
> --------------------------------------
>
> Key: USERGRID-255
> URL: https://issues.apache.org/jira/browse/USERGRID-255
> Project: Usergrid
> Issue Type: Story
> Components: Stack
> Reporter: Todd Nine
> Assignee: David Johnson
>
> The two-dot-o Query Index module currently stores both document source and
> fields in ElasticSearch. Since we only ever retrieve ID numbers from ES,
> there is no need for us to store source and it is a waste of resources.
> This is what we need:
> 1) A way to configure Usergrid to either store source or not store source.
> 2) An index-rebuild "Tool" (implemented as a REST end-point) that either
> remove source, or add source depending on how the system in configured to
> operate. The Tool must allow us to re-index without downtime. Possible
> approach:
> For each application:
> a) Tool creates a new index and adds that index to the application's read
> and write alias.
> b) Tool removes the old index from the application's write alias so it is
> no longer written to.
> c) Tool deletes the mappings for each newly added index, then re-creates
> them with the new store-source settings.
> d) Tool re-indexes the application's collections.
> e) Once re-index is complete, Tool deletes the old index
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
