[ https://issues.apache.org/jira/browse/CASSANDRA-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890762#action_12890762 ]
Nirmal Ranganathan commented on CASSANDRA-1189: ----------------------------------------------- Here's some proposed changes, please comment with feedback. There are two occurrences of streaming: Source transfers to Destination (Anti-entropy repair, node decommission, possibly bulk import) - In each of the cases source has a list of sstable files it needs to transfer to the destination. - Source maintains a list of all the files, source creates a session id for transferring this set of files. - Source streams the first file, header contains a new StreamHeader, that has the PendingFile info embedded. - Destination receives the stream, it has all the info for the file, once done responds with a StreamStatus message. - If StreamStatus is success, Source continues with next file, if not retransfer until all files are complete. (Approach 1) Destination requests from Source (Anti-entropy repair, bootstrap, possibly bulk export) - Destination complies list of ranges and sends a StreamRequest message to Source, it attaches a session id to keep track of the request. - Source based on the ranges compiles a list of PendingFile's and sends a StreamRequestResponse message with the list of files. - Destination now has the list of files to maintain state. - Destination sends a StreamRequest for a file from the list, it has a session id and file descriptor info attached. - Source Streams the file to Destination. - Destination based on the transfer status, requests the next file or re-requests the same file, until all files are transferred. (Approach 2) Destination requests from Source (Anti-entropy repair, bootstrap, possibly bulk export) - Destination complies list of ranges and sends a StreamRequest message to Source, it attaches a session id to keep track of the request. - Source compiles list of PendingFile's from requested ranges. Source maintains state. - Source Streams file 1 with attached StreamHeader. - Destination receives file and responds with a StreamStatus. - Source based on status transfers the next file or re-transfers the same file. Changes to Protocol for File Streaming: - Current -> | Protocol magic | Header | Body (File contents) | - Proposed -> | Protocol magic | Header | StreamHeader size | StreamHeader | Body (File contents) | - The protocol for all other Message's remain the same, the format remains the same, the content will vary. Effects of the mentioned changes: - There can be multiple transfers per source and destination. - No order of files is required, prevents overlapping streams from breaking anything. - Other services can transfer files without a problem. - Initiate and Initiate Done will be removed. A little cleaner process. - Facilitates for adding a layer on top to do bulk imports/exports. Questions: - The current streaming does not seem to maintain persistant state if a node fails during streaming, would that be something that needs to be considered. - Do we want to add checksums? > Refactor streaming > ------------------ > > Key: CASSANDRA-1189 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1189 > Project: Cassandra > Issue Type: Improvement > Components: Core > Affects Versions: 0.7 > Reporter: Gary Dusbabek > Assignee: Nirmal Ranganathan > Priority: Critical > Fix For: 0.7 > > > The current architecture is buggy because it makes the assumption that only > one stream can be in process between two nodes at a given time, and stream > send order never changes. Because of this, the ACK process gets fouled up > when other services wish to stream files. > The process is somewhat contorted too (request, initiate, initiate done, > send). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.