[
https://issues.apache.org/jira/browse/ATLAS-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-4340:
-----------------------------------
Description:
In Atlas when a transaction is committed, the entries are committed to HBase
(primary storage) and Solr (indexing storage). A transaction is rolled-back if
the primary storage commit fails, on the other hand when the secondary commit
fails (solr), the transaction is not-rolled back and logged as warning and it
is recommended to use reindex to repair the missing index documents. This
behavior is due to the fact that the primary storage is the source of truth and
indexes can be rebuild.
In Janusgraph, there is a property for Solr to make solr commits async. This is
set to *true* in Atlas making every commit to wait until the solr commit is
successful. This will have a negative impact on performance and is recommended
to be false by default.
Property: *index.[X].solr.wait-searcher*
|When mutating - wait for the index to reflect new mutations before returning.
This can have a negative impact on performance.|
This Jira is about setting the default value for above property to FALSE and
can be overridden if need arises.
The solution should use the _StandardTransactionLogProcessor_ provided within
JanusGraph to track failures to indexes (secondary storage in JanusGraph
parlance) during commit. Using this would provide recovery mechanism in case of
failures during transaction commit.
was:
In Atlas when a transaction is committed, the entries are committed to HBase
(primary storage) and Solr (indexing storage). A transaction is rolled-back if
the primary storage commit fails, on the other hand when the secondary commit
fails (solr), the transaction is not-rolled back and logged as warning and it
is recommended to use reindex to repair the missing index documents. This
behavior is due to the fact that the primary storage is the source of truth and
indexes can be rebuild.
In Janusgraph, there is a property for Solr to make solr commits async. This is
set to *true* in Atlas making every commit to wait until the solr commit is
successful. This will have a negative impact on performance and is recommended
to be false by default.
Property: *index.[X].solr.wait-searcher*
|When mutating - wait for the index to reflect new mutations before returning.
This can have a negative impact on performance.|
This Jira is about setting the default value for above property to FALSE and
can be overridden if need arises.
> Set Solr wait-searcher property to false by default to make Solr commits async
> ------------------------------------------------------------------------------
>
> Key: ATLAS-4340
> URL: https://issues.apache.org/jira/browse/ATLAS-4340
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 2.1.0
> Reporter: Sarath Subramanian
> Assignee: Sarath Subramanian
> Priority: Major
> Labels: perfomance, solr
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4340-001.patch
>
>
> In Atlas when a transaction is committed, the entries are committed to HBase
> (primary storage) and Solr (indexing storage). A transaction is rolled-back
> if the primary storage commit fails, on the other hand when the secondary
> commit fails (solr), the transaction is not-rolled back and logged as warning
> and it is recommended to use reindex to repair the missing index documents.
> This behavior is due to the fact that the primary storage is the source of
> truth and indexes can be rebuild.
> In Janusgraph, there is a property for Solr to make solr commits async. This
> is set to *true* in Atlas making every commit to wait until the solr commit
> is successful. This will have a negative impact on performance and is
> recommended to be false by default.
> Property: *index.[X].solr.wait-searcher*
> |When mutating - wait for the index to reflect new mutations before
> returning. This can have a negative impact on performance.|
>
> This Jira is about setting the default value for above property to FALSE and
> can be overridden if need arises.
> The solution should use the _StandardTransactionLogProcessor_ provided within
> JanusGraph to track failures to indexes (secondary storage in JanusGraph
> parlance) during commit. Using this would provide recovery mechanism in case
> of failures during transaction commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
