[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Kjellman updated CASSANDRA-4813: Comment: was deleted (was: [~yukim] I take it back. Ship It! Dependency issues on my end with the last patch, sorry for the false alarm.) > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 rc1 > > Attachments: 4813.txt > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4813: -- Attachment: (was: 4813.txt) > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 rc1 > > Attachments: 4813.txt > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4813: -- Attachment: 4813.txt ok, it looks like we have CASSANDRA-3839 for BOF. Updated patch to avoid socket re-binding. [~mkjellman] How about this one? > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 rc1 > > Attachments: 4813.txt > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4813: -- Attachment: 4813.txt Attaching newer version. Since with this patch, we only distinguish streaming session by UUID, we don't need to carry around broadcast address(CASSANDRA-3503), so I removed it from StreamHeader. > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 > > Attachments: 4813.txt > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4813: -- Attachment: (was: 4813.txt) > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-4813: -- Attachment: 4813.txt Attaching patch to change streaming session ID from (host, counter) pair to Time UUID. This should hugely drop probability of session ID collision. I haven't tested with BOF, but I spawned 3 sstableloader simultaneously on the same node with C* and could finish streaming without getting errors. > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 > > Attachments: 4813.txt > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-4813: -- Priority: Minor (was: Major) Affects Version/s: (was: 1.1.5) (was: 1.1.3) 1.1.0 Fix Version/s: 1.2.0 If we need to change streaming protocol to fix this then we should target 1.2. > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.0 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos >Assignee: Yuki Morishita >Priority: Minor > Labels: Bulkoutputformat, Hadoop, SSTables > Fix For: 1.2.0 > > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4813) Problem using BulkOutputFormat while streaming several SSTables simultaneously from a given node.
[ https://issues.apache.org/jira/browse/CASSANDRA-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Romanos updated CASSANDRA-4813: - Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2. (was: I am using SLES 10 SP3, 4 Cassandra + Hadoop nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using Hadoop 0.20.2.) > Problem using BulkOutputFormat while streaming several SSTables > simultaneously from a given node. > - > > Key: CASSANDRA-4813 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4813 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.1.3, 1.1.5 > Environment: I am using SLES 10 SP3, Java 6, 4 Cassandra + Hadoop > nodes, 3 Hadoop only nodes (datanodes/tasktrackers), 1 namenode/jobtracker. > The machines used are Six-Core AMD Opteron(tm) Processor 8431, 24 cores and > 33 GB of RAM. I get the issue on both cassandra 1.1.3, 1.1.5 and I am using > Hadoop 0.20.2. >Reporter: Ralph Romanos > Labels: Bulkoutputformat, Hadoop, SSTables > > The issue occurs when streaming simultaneously SSTables from the same node to > a cassandra cluster using SSTableloader. It seems to me that Cassandra cannot > handle receiving simultaneously SSTables from the same node. However, when it > receives simultaneously SSTables from two different nodes, everything works > fine. As a consequence, when using BulkOutputFormat to generate SSTables and > stream them to a cassandra cluster, I cannot use more than one reducer per > node otherwise I get a java.io.EOFException in the tasktracker's logs and a > java.io.IOException: Broken pipe in the Cassandra logs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
