[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Fix Version/s: (was: 2.0) 2.0 beta 2 BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 beta 2 Attachments: 0001-change-streaming-procedure.patch, 0002-make-BulkLoader-work-with-SSTableReader.patch, 0003-update-for-post-CASSANDRA-5171.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: (was: 0001-change-streaming-procedure.patch) BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 beta 2 Attachments: 0002-make-BulkLoader-work-with-SSTableReader.patch, 0003-update-for-post-CASSANDRA-5171.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: 0001-change-streaming-procedure.patch Thanks for the review. Updated 0001 patch. bq. Does separating initiated and receiving stream in StreamManager really buys us anything? (outside slightly complicating things) We need to keep StreamResultFuture of the same plan ID when we are doing streaming on the same JVM. This happens when we bulk load via JMX method, or running StreamingTransferTest unit test. bq. On failure (StreamSession.onError), we could (and probably should) send a message as long as the outgoing connection is opened (even if the incoming one is broken), so the current isConnected() is possibly a bit restrictive. You are right. I changed to put null check on outgoing message handler before sending message instead. BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 beta 2 Attachments: 0001-change-streaming-procedure.patch, 0002-make-BulkLoader-work-with-SSTableReader.patch, 0003-update-for-post-CASSANDRA-5171.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: (was: 0002-make-BulkLoader-work-with-SSTableReader.patch) BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: (was: 0001-change-streaming-procedure.patch) BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: 0001-change-streaming-procedure.patch 0002-make-BulkLoader-work-with-SSTableReader.patch 0003-update-for-post-CASSANDRA-5171.patch Uploaded update patches. * 0001 changes so that streaming initiator makes two connections. It also changes how StreamManager manages current connection in order to distinguish plans inside the same JVM (for bulkload JMX op and unit test). * 0002 is updated for sstableloader command output. * 0003 introduces change in how we create socket from streaming session. CASSANDRA-5171 makes OutboundTCPConnection pool require system table which client like sstableloader doesn't have. Instead, I made newSocket static method so we just can obtain socket for specific host. BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 0001-change-streaming-procedure.patch, 0002-make-BulkLoader-work-with-SSTableReader.patch, 0003-update-for-post-CASSANDRA-5171.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5542: -- Reviewer: slebresne (was: jbellis) BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 0001-change-streaming-procedure.patch, 0002-make-BulkLoader-work-with-SSTableReader.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: 0001-change-streaming-procedure.patch 0002-make-BulkLoader-work-with-SSTableReader.patch Thanks Sylvain. Attaching 2 patches, 0001 changes streaming 2.0 procedure based on Sylvain's comment above(initiator opens 2 connections), and 0002 to query schema to open SSTableReader for streaming. BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 beta 1 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 0001-change-streaming-procedure.patch, 0002-make-BulkLoader-work-with-SSTableReader.patch, 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: 5542-fix-NPE.txt Attaching patch to switch using indexSummary instead of CFMetaData where affected. BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 Reporter: Yuki Morishita Assignee: Yuki Morishita Fix For: 2.0 Attachments: 5542-fix-NPE.txt, 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-5542: -- Attachment: 5542.txt There are two classes that need to change: * DatabaseDescriptor We need to set memoryAllocator in order to create IndexSummary. * SSTableLoader Create dummy CFMetadata and pass it to SSTableReader.open, so it can use default values for bf_fp_chance and index_interval. I'm not sure if that the best approach for second one though. BulkLoader is broken in trunk - Key: CASSANDRA-5542 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542 Project: Cassandra Issue Type: Bug Affects Versions: 2.0 Reporter: Yuki Morishita Attachments: 5542.txt After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval. When using bulkloader, CFMetaData is not available, so we cannot open SSTable to be streamed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
