[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Issue Type: Improvement (was: Sub-task) Parent: (was: IGNITE-3477) > Introduce persistence interface to allow build reliable persistence plugins > --- > > Key: IGNITE-4337 > URL: https://issues.apache.org/jira/browse/IGNITE-4337 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk > Labels: important > Fix For: 2.1 > > > With page memory interface introduced, it may be possible to build a > persistence layer around this architecture. I think we should move the > PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package > and introduce the following interface to allow other components to log it's > activity in crash-resistant way: > {code} > /** > * > */ > public interface IgniteWriteAheadLogManager extends GridCacheSharedManager { > /** > * @return {@code true} If we have to always write full pages. > */ > public boolean isAlwaysWriteFullPages(); > /** > * @return {@code true} if WAL will perform fair syncs on fsync call. > */ > public boolean isFullSync(); > /** > * Resumes logging after start. When WAL manager is started, it will skip > logging any updates until this > * method is called to avoid logging changes induced by the state restore > procedure. > */ > public void resumeLogging(WALPointer lastWrittenPtr) throws > IgniteCheckedException; > /** > * Appends the given log entry to the write-ahead log. > * > * @param entry entry to log. > * @return WALPointer that may be passed to {@link #fsync(WALPointer)} > method to make sure the record is > * written to the log. > * @throws IgniteCheckedException If failed to construct log entry. > * @throws StorageException If IO error occurred while writing log entry. > */ > public WALPointer log(WALRecord entry) throws IgniteCheckedException, > StorageException; > /** > * Makes sure that all log entries written to the log up until the > specified pointer are actually persisted to > * the underlying storage. > * > * @param ptr Optional pointer to sync. If {@code null}, will sync up to > the latest record. > * @throws IgniteCheckedException If > * @throws StorageException > */ > public void fsync(WALPointer ptr) throws IgniteCheckedException, > StorageException; > /** > * Invoke this method to iterate over the written log entries. > * > * @param start Optional WAL pointer from which to start iteration. > * @return Records iterator. > * @throws IgniteException If failed to start iteration. > * @throws StorageException If IO error occurred while reading WAL > entries. > */ > public WALIterator replay(WALPointer start) throws > IgniteCheckedException, StorageException; > /** > * Gives a hint to WAL manager to clear entries logged before the given > pointer. Some entries before the > * the given pointer will be kept because there is a configurable WAL > history size. Those entries may be used > * for partial partition rebalancing. > * > * @param ptr Pointer for which it is safe to clear the log. > * @return Number of deleted WAL segments. > */ > public int truncate(WALPointer ptr); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Fix Version/s: (was: 2.0) 2.1 > Introduce persistence interface to allow build reliable persistence plugins > --- > > Key: IGNITE-4337 > URL: https://issues.apache.org/jira/browse/IGNITE-4337 > Project: Ignite > Issue Type: Sub-task > Components: general >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk > Labels: important > Fix For: 2.1 > > > With page memory interface introduced, it may be possible to build a > persistence layer around this architecture. I think we should move the > PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package > and introduce the following interface to allow other components to log it's > activity in crash-resistant way: > {code} > /** > * > */ > public interface IgniteWriteAheadLogManager extends GridCacheSharedManager { > /** > * @return {@code true} If we have to always write full pages. > */ > public boolean isAlwaysWriteFullPages(); > /** > * @return {@code true} if WAL will perform fair syncs on fsync call. > */ > public boolean isFullSync(); > /** > * Resumes logging after start. When WAL manager is started, it will skip > logging any updates until this > * method is called to avoid logging changes induced by the state restore > procedure. > */ > public void resumeLogging(WALPointer lastWrittenPtr) throws > IgniteCheckedException; > /** > * Appends the given log entry to the write-ahead log. > * > * @param entry entry to log. > * @return WALPointer that may be passed to {@link #fsync(WALPointer)} > method to make sure the record is > * written to the log. > * @throws IgniteCheckedException If failed to construct log entry. > * @throws StorageException If IO error occurred while writing log entry. > */ > public WALPointer log(WALRecord entry) throws IgniteCheckedException, > StorageException; > /** > * Makes sure that all log entries written to the log up until the > specified pointer are actually persisted to > * the underlying storage. > * > * @param ptr Optional pointer to sync. If {@code null}, will sync up to > the latest record. > * @throws IgniteCheckedException If > * @throws StorageException > */ > public void fsync(WALPointer ptr) throws IgniteCheckedException, > StorageException; > /** > * Invoke this method to iterate over the written log entries. > * > * @param start Optional WAL pointer from which to start iteration. > * @return Records iterator. > * @throws IgniteException If failed to start iteration. > * @throws StorageException If IO error occurred while reading WAL > entries. > */ > public WALIterator replay(WALPointer start) throws > IgniteCheckedException, StorageException; > /** > * Gives a hint to WAL manager to clear entries logged before the given > pointer. Some entries before the > * the given pointer will be kept because there is a configurable WAL > history size. Those entries may be used > * for partial partition rebalancing. > * > * @param ptr Pointer for which it is safe to clear the log. > * @return Number of deleted WAL segments. > */ > public int truncate(WALPointer ptr); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Magda updated IGNITE-4337: Labels: important (was: ) > Introduce persistence interface to allow build reliable persistence plugins > --- > > Key: IGNITE-4337 > URL: https://issues.apache.org/jira/browse/IGNITE-4337 > Project: Ignite > Issue Type: Sub-task > Components: general >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk > Labels: important > Fix For: 2.0 > > > With page memory interface introduced, it may be possible to build a > persistence layer around this architecture. I think we should move the > PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package > and introduce the following interface to allow other components to log it's > activity in crash-resistant way: > {code} > /** > * > */ > public interface IgniteWriteAheadLogManager extends GridCacheSharedManager { > /** > * @return {@code true} If we have to always write full pages. > */ > public boolean isAlwaysWriteFullPages(); > /** > * @return {@code true} if WAL will perform fair syncs on fsync call. > */ > public boolean isFullSync(); > /** > * Resumes logging after start. When WAL manager is started, it will skip > logging any updates until this > * method is called to avoid logging changes induced by the state restore > procedure. > */ > public void resumeLogging(WALPointer lastWrittenPtr) throws > IgniteCheckedException; > /** > * Appends the given log entry to the write-ahead log. > * > * @param entry entry to log. > * @return WALPointer that may be passed to {@link #fsync(WALPointer)} > method to make sure the record is > * written to the log. > * @throws IgniteCheckedException If failed to construct log entry. > * @throws StorageException If IO error occurred while writing log entry. > */ > public WALPointer log(WALRecord entry) throws IgniteCheckedException, > StorageException; > /** > * Makes sure that all log entries written to the log up until the > specified pointer are actually persisted to > * the underlying storage. > * > * @param ptr Optional pointer to sync. If {@code null}, will sync up to > the latest record. > * @throws IgniteCheckedException If > * @throws StorageException > */ > public void fsync(WALPointer ptr) throws IgniteCheckedException, > StorageException; > /** > * Invoke this method to iterate over the written log entries. > * > * @param start Optional WAL pointer from which to start iteration. > * @return Records iterator. > * @throws IgniteException If failed to start iteration. > * @throws StorageException If IO error occurred while reading WAL > entries. > */ > public WALIterator replay(WALPointer start) throws > IgniteCheckedException, StorageException; > /** > * Gives a hint to WAL manager to clear entries logged before the given > pointer. Some entries before the > * the given pointer will be kept because there is a configurable WAL > history size. Those entries may be used > * for partial partition rebalancing. > * > * @param ptr Pointer for which it is safe to clear the log. > * @return Number of deleted WAL segments. > */ > public int truncate(WALPointer ptr); > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Description: With page memory interface introduced, it may be possible to build a persistence layer around this architecture. I think we should move the PageMemory interface itself to {{org.apache.ignite.plugin.storage}} package and introduce the following interface to allow other components to log it's activity in crash-resistant way: {code} /** * */ public interface IgniteWriteAheadLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if WAL will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional WAL pointer from which to start iteration. * @return Records iterator. * @throws IgniteException If failed to start iteration. * @throws StorageException If IO error occurred while reading WAL entries. */ public WALIterator replay(WALPointer start) throws IgniteCheckedException, StorageException; /** * Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the * the given pointer will be kept because there is a configurable WAL history size. Those entries may be used * for partial partition rebalancing. * * @param ptr Pointer for which it is safe to clear the log. * @return Number of deleted WAL segments. */ public int truncate(WALPointer ptr); } {code} was: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if persistence manager will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When persistence manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(LogPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return LogPointer that may be passed to {@link #fsync(LogPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public LogPointer log(LogRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(LogPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional log pointer fr
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Description: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if persistence manager will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When persistence manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(LogPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return LogPointer that may be passed to {@link #fsync(LogPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public LogPointer log(LogRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(LogPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional log pointer from which to start iteration. * @return Records iterator. * @throws IgniteException If failed to start iteration. * @throws StorageException If IO error occurred while reading log entries. */ public LogIterator replay(LogPointer start) throws IgniteCheckedException, StorageException; /** * Gives a hint to persistence manager to clear entries logged before the given pointer. Some entries before the * the given pointer will be kept because there is a configurable log history size. Those entries may be used * for partial partition rebalancing. * * @param ptr Pointer for which it is safe to clear the log. * @return Number of deleted log segments. */ public int truncate(LogPointer ptr); } {code} was: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if persistence manager will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When persistence manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(LogPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return LogPointer that may be passed to {@link #fsync(LogPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public LogPointer log(LogRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(LogPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional log pointer from which to start iteration. * @return Records iterator.
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Description: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if persistence manager will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When persistence manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(LogPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return LogPointer that may be passed to {@link #fsync(LogPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public LogPointer log(LogRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(LogPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional log pointer from which to start iteration. * @return Records iterator. * @throws IgniteException If failed to start iteration. * @throws StorageException If IO error occurred while reading WAL entries. */ public LogIterator replay(LogPointer start) throws IgniteCheckedException, StorageException; /** * Gives a hint to persistence manager to clear entries logged before the given pointer. Some entries before the * the given pointer will be kept because there is a configurable log history size. Those entries may be used * for partial partition rebalancing. * * @param ptr Pointer for which it is safe to clear the log. * @return Number of deleted log segments. */ public int truncate(LogPointer ptr); } {code} was: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if WAL will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional WAL pointer from which to start iteration. * @return Records iterator. * @throws IgniteExceptio
[jira] [Updated] (IGNITE-4337) Introduce persistence interface to allow build reliable persistence plugins
[ https://issues.apache.org/jira/browse/IGNITE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-4337: - Description: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgnitePersistenceLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if WAL will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional WAL pointer from which to start iteration. * @return Records iterator. * @throws IgniteException If failed to start iteration. * @throws StorageException If IO error occurred while reading WAL entries. */ public WALIterator replay(WALPointer start) throws IgniteCheckedException, StorageException; /** * Gives a hint to WAL manager to clear entries logged before the given pointer. Some entries before the * the given pointer will be kept because there is a configurable WAL history size. Those entries may be used * for partial partition rebalancing. * * @param ptr Pointer for which it is safe to clear the log. * @return Number of deleted WAL segments. */ public int truncate(WALPointer ptr); } {code} was: If page memory interface is introduced, it may be possible to build a persistence layer around this architecture. I think we should add some form of persistence logging to allow us build a crash-resistant system in future. Something like {code} public interface IgniteWriteAheadLogManager extends GridCacheSharedManager { /** * @return {@code true} If we have to always write full pages. */ public boolean isAlwaysWriteFullPages(); /** * @return {@code true} if WAL will perform fair syncs on fsync call. */ public boolean isFullSync(); /** * Resumes logging after start. When WAL manager is started, it will skip logging any updates until this * method is called to avoid logging changes induced by the state restore procedure. */ public void resumeLogging(WALPointer lastWrittenPtr) throws IgniteCheckedException; /** * Appends the given log entry to the write-ahead log. * * @param entry entry to log. * @return WALPointer that may be passed to {@link #fsync(WALPointer)} method to make sure the record is * written to the log. * @throws IgniteCheckedException If failed to construct log entry. * @throws StorageException If IO error occurred while writing log entry. */ public WALPointer log(WALRecord entry) throws IgniteCheckedException, StorageException; /** * Makes sure that all log entries written to the log up until the specified pointer are actually persisted to * the underlying storage. * * @param ptr Optional pointer to sync. If {@code null}, will sync up to the latest record. * @throws IgniteCheckedException If * @throws StorageException */ public void fsync(WALPointer ptr) throws IgniteCheckedException, StorageException; /** * Invoke this method to iterate over the written log entries. * * @param start Optional WAL pointer from which to start iteration. * @return Records iterator. * @throws IgniteException If failed to start iteration.