Re: Ignite transaction recovery on third-party persistence
SQL APIs are not transactional yet. You need to use key-value calls within Ignite transactions if ACID guarantees are required. SQL will become fully transactional once MVCC becomes ready for the GA release. https://apacheignite.readme.io/docs/multiversion-concurrency-control - Denis On Wed, Sep 25, 2019 at 5:28 AM bijunathg wrote: > Thanks Denis & Ilya! > We found an issue if the transaction coordinator exits after successfully > committing to the third-party store, but before propagating the commit to > the Ignite server nodes in the cluster. We observe two behaviors: > 1. *If the committed transaction had insert() statements, those entries are > not available in cache* > 2. If the committed transaction had update() statements, those updated > entries are correctly available in cache. > > Any explanation for this behavior? > > Note: Our CacheAtomicityMode is set as TRANSACTIONAL and > CacheWriteSynchronizationMode is set as FULL_SYNC. Also our application > pre-loads all entries in cache at startup and we serve SQL queries from > cache. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Ignite transaction recovery on third-party persistence
Thanks Denis & Ilya! We found an issue if the transaction coordinator exits after successfully committing to the third-party store, but before propagating the commit to the Ignite server nodes in the cluster. We observe two behaviors: 1. *If the committed transaction had insert() statements, those entries are not available in cache* 2. If the committed transaction had update() statements, those updated entries are correctly available in cache. Any explanation for this behavior? Note: Our CacheAtomicityMode is set as TRANSACTIONAL and CacheWriteSynchronizationMode is set as FULL_SYNC. Also our application pre-loads all entries in cache at startup and we serve SQL queries from cache. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite transaction recovery on third-party persistence
Minor clarification. The updates will be fully consistent in case if a transaction commits to Ignite Memory + RBDMS or Ignite Memory + Native Persistence. There are no guarantees if the transaction updates both RDBMS or Native Persistence. - Denis On Thu, Sep 5, 2019 at 7:03 AM Ilya Kasnacheev wrote: > Hello! > > Transaction failure and recovery scenarios have no relation with Native > Persistence. Transactional safety is guaranteed irrespective of Native > persistence setting. > > However, since Cache Store does not support two phase commit, there may be > divergence between Native Persistence and Cache Store, by definition. > > Regards, > -- > Ilya Kasnacheev > > > вт, 3 сент. 2019 г. в 14:36, bijunathg : > >> Hi, >> >> We have strict transactional requirements to query and update some cached >> data. Our queries are mostly SQL and so we fully load data at application >> startup. Our cached data is huge so we use a partitioned cache for >> scalability. >> >> Also for some BI applications, we need to write the data into an RDBMS >> persistent store and make it available in a consistent manner. So we are >> planning to use Ignite's write-through mode which updates both the DB and >> cache atomically within an Ignite transaction. >> >> But while exploring the Ignite documentation on third-party persistence >> and >> transaction management >> ( >> https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party >> ), >> looks like transaction failure & recovery scenarios (like DB write >> successful, but cache commit failed) work only if Native persistence is >> enabled. Since we need SQL queries, the read-through approach may not >> work, >> as per our understanding. >> >> Should we enable both Ignite native persistence for High availability and >> third-party persistence for our BI requirements? Will this ensure that the >> SQL query results are always consistent? >> >> Thanks, >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >
Re: Ignite transaction recovery on third-party persistence
Hello! Transaction failure and recovery scenarios have no relation with Native Persistence. Transactional safety is guaranteed irrespective of Native persistence setting. However, since Cache Store does not support two phase commit, there may be divergence between Native Persistence and Cache Store, by definition. Regards, -- Ilya Kasnacheev вт, 3 сент. 2019 г. в 14:36, bijunathg : > Hi, > > We have strict transactional requirements to query and update some cached > data. Our queries are mostly SQL and so we fully load data at application > startup. Our cached data is huge so we use a partitioned cache for > scalability. > > Also for some BI applications, we need to write the data into an RDBMS > persistent store and make it available in a consistent manner. So we are > planning to use Ignite's write-through mode which updates both the DB and > cache atomically within an Ignite transaction. > > But while exploring the Ignite documentation on third-party persistence and > transaction management > ( > https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party > ), > looks like transaction failure & recovery scenarios (like DB write > successful, but cache commit failed) work only if Native persistence is > enabled. Since we need SQL queries, the read-through approach may not work, > as per our understanding. > > Should we enable both Ignite native persistence for High availability and > third-party persistence for our BI requirements? Will this ensure that the > SQL query results are always consistent? > > Thanks, > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Ignite transaction recovery on third-party persistence
Hi, We have strict transactional requirements to query and update some cached data. Our queries are mostly SQL and so we fully load data at application startup. Our cached data is huge so we use a partitioned cache for scalability. Also for some BI applications, we need to write the data into an RDBMS persistent store and make it available in a consistent manner. So we are planning to use Ignite's write-through mode which updates both the DB and cache atomically within an Ignite transaction. But while exploring the Ignite documentation on third-party persistence and transaction management (https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party), looks like transaction failure & recovery scenarios (like DB write successful, but cache commit failed) work only if Native persistence is enabled. Since we need SQL queries, the read-through approach may not work, as per our understanding. Should we enable both Ignite native persistence for High availability and third-party persistence for our BI requirements? Will this ensure that the SQL query results are always consistent? Thanks, -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
