[ https://issues.apache.org/jira/browse/CASSANDRA-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249126#comment-13249126 ]
Vijay commented on CASSANDRA-3690: ---------------------------------- Hi Jonathan, Agree, would you like to remove the configuration to disable recycle (which this patch added)? let me know... Thanks! > Streaming CommitLog backup > -------------------------- > > Key: CASSANDRA-3690 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3690 > Project: Cassandra > Issue Type: Bug > Components: Tools > Reporter: Vijay > Assignee: Vijay > Priority: Minor > Fix For: 1.1.1 > > Attachments: 0001-CASSANDRA-3690-v2.patch, > 0001-CASSANDRA-3690-v4.patch, 0001-Make-commitlog-recycle-configurable.patch, > 0002-support-commit-log-listener.patch, 0003-helper-jmx-methods.patch, > 0004-external-commitlog-with-sockets.patch, > 0005-cmmiting-comments-to-yaml.patch > > > Problems with the current SST backups > 1) The current backup doesn't allow us to restore point in time (within a SST) > 2) Current SST implementation needs the backup to read from the filesystem > and hence additional IO during the normal operational Disks > 3) in 1.0 we have removed the flush interval and size when the flush will be > triggered per CF, > For some use cases where there is less writes it becomes > increasingly difficult to time it right. > 4) Use cases which needs BI which are external (Non cassandra), needs the > data in regular intervals than waiting for longer or unpredictable intervals. > Disadvantages of the new solution > 1) Over head in processing the mutations during the recover phase. > 2) More complicated solution than just copying the file to the archive. > Additional advantages: > Online and offline restore. > Close to live incremental backup. > Note: If the listener agent gets restarted, it is the agents responsibility > to Stream the files missed or incomplete. > There are 3 Options in the initial implementation: > 1) Backup -> Once a socket is connected we will switch the commit log and > send new updates via the socket. > 2) Stream -> will take the absolute path of the file and will read the file > and send the updates via the socket. > 3) Restore -> this will get the serialized bytes and apply's the mutation. > Side NOTE: (Not related to this patch as such) The agent which will take > incremental backup is planned to be open sourced soon (Name: Priam). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira