[jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764273#comment-13764273
]
Henrik Baastrup commented on CASSANDRA-5987:
I have been debugging Cassandra Daemon, and as fare I can see, the problem is
in SSTableWriter class.
At SSTableWriter:234 when the Atom iterator is created, the OnDiskAtom:74 reads
the name field correctly from the file, using the two length bytes in front of
the name string (the length bytes are not stored in the name buffer), later in
SSTableWriter:254 when the minColumnNames is calculated, the atom name is now
passed, without the length bytes, but when AbstractCompositeType:130 does the
split it expect the length bytes at AbstractCompositeType:63.
The question is if it is necessary to do the split or the name can be used as
is.
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
> at
> org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
> at
> org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
> at
> org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
> ... 4 more
> INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862
> StreamResultFuture.java (line 180) [Stream
> #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
> WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864
> StreamResultFuture.java (line 2
[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763092#comment-13763092
]
Henrik Baastrup edited comment on CASSANDRA-5987 at 9/10/13 3:00 PM:
-
Hi Yuki,
I see your point! I have now simplified my program and table so I do not use
composite keys, but only one primary key; I use a UUID I generate on the fly
for this.
The problem remain, I still get the IllegalArgumentException.
I changed my create table statement as follow:
{quote}
CREATE TABLE nsmo_dns (
key uuid PRIMARY KEY,
endtime bigint,
lastframetime bigint,
.
.
{quote}
no PRIMARY KEY (callsequencenumber,moduleid).
My SSTableSimpleUnsortedWriter create statement is as follow:
{quote}
new SSTableSimpleUnsortedWriter(
directory,
partitioner,
keyspace,
columnFamily,
UUIDType.instance,
null,
BUFFER_SIZE);
{quote}
My write looks like this now:
{quote}
public void write(SSTableSimpleUnsortedWriter writer, long timestamp) throws
IOException {
writer.newRow(ByteBuffer.wrap(decompose(key)));
writer.addColumn(bytes("endtime"), bytes(endtime), timestamp);
writer.addColumn(bytes("lastframetime"), bytes(lastframetime), timestamp);
writer.addColumn(bytes("begintime"), bytes(begintime), timestamp);
.
.
{quote}
As I see it, either does SSTableSimpleUnsortedWriter not write the length of
the column name in front of it, or the AbstractCompositeType in the Cassandra
engine expect a length of the column name where there is none.
Of cause, there exist also the possibility that I have missed something whats
very likely ;)
was (Author: hbaastrup):
Hi Yuki,
I see your point! I have now simplified my program and table so I do not use
composite keys, but only one primary key; I use a UUID I generate on the fly
for this.
The problem remain, I still get the IllegalArgumentException.
I changed my create table statement as follow:
{quote}
CREATE TABLE nsmo_dns (
key uuid PRIMARY KEY,
endtime bigint,
lastframetime bigint,
.
.
{quote}
and my SSTableSimpleUnsortedWriter create statement is as follow:
{quote}
new SSTableSimpleUnsortedWriter(
directory,
partitioner,
keyspace,
columnFamily,
UUIDType.instance,
null,
BUFFER_SIZE);
{quote}
My write looks like this now:
{quote}
public void write(SSTableSimpleUnsortedWriter writer, long timestamp) throws
IOException {
writer.newRow(ByteBuffer.wrap(decompose(key)));
writer.addColumn(toBufferWithShortLength("endtime"), bytes(endtime),
timestamp);
writer.addColumn(toBufferWithShortLength("lastframetime"),
bytes(lastframetime), timestamp);
writer.addColumn(bytes("begintime"), bytes(begintime), timestamp);
.
.
{quote}
As I see it, either does SSTableSimpleUnsortedWriter not write the length of
the column name in front of it, or the AbstractCompositeType in the Cassandra
engine expect a length of the column name where there is none.
Of cause, there exist also the possibility that I have missed something ;)
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-194
[jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13763092#comment-13763092
]
Henrik Baastrup commented on CASSANDRA-5987:
Hi Yuki,
I see your point! I have now simplified my program and table so I do not use
composite keys, but only one primary key; I use a UUID I generate on the fly
for this.
The problem remain, I still get the IllegalArgumentException.
I changed my create table statement as follow:
{quote}
CREATE TABLE nsmo_dns (
key uuid PRIMARY KEY,
endtime bigint,
lastframetime bigint,
.
.
{quote}
and my SSTableSimpleUnsortedWriter create statement is as follow:
{quote}
new SSTableSimpleUnsortedWriter(
directory,
partitioner,
keyspace,
columnFamily,
UUIDType.instance,
null,
BUFFER_SIZE);
{quote}
My write looks like this now:
{quote}
public void write(SSTableSimpleUnsortedWriter writer, long timestamp) throws
IOException {
writer.newRow(ByteBuffer.wrap(decompose(key)));
writer.addColumn(toBufferWithShortLength("endtime"), bytes(endtime),
timestamp);
writer.addColumn(toBufferWithShortLength("lastframetime"),
bytes(lastframetime), timestamp);
writer.addColumn(bytes("begintime"), bytes(begintime), timestamp);
.
.
{quote}
As I see it, either does SSTableSimpleUnsortedWriter not write the length of
the column name in front of it, or the AbstractCompositeType in the Cassandra
engine expect a length of the column name where there is none.
Of cause, there exist also the possibility that I have missed something ;)
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
> at
> org.apache.cassandra.io.sstable.ColumnNameHel
[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761882#comment-13761882
]
Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:31 PM:
Attached my program there create the SSTables. Set-up the classpath to contain
all necessary Cassandra jars and lunch the program like this:
*java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf*
Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}
was (Author: hbaastrup):
Attached my program there create the SSTables. Set-up the classpath to
contain all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassan
[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761882#comment-13761882
]
Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:30 PM:
Attached my program there create the SSTables. Set-up the classpath to contain
all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}
was (Author: hbaastrup):
Attached my program there create the SSTables. Set-up the classpath to
contain all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name nsmo_ds with the following table:
REATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.Connecti
[jira] [Updated] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henrik Baastrup updated CASSANDRA-5987:
---
Attachment: CassandraLoader.tgz
Attached my program there create the SSTables. Set-up the classpath to contain
all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name nsmo_ds with the following table:
REATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
> at
> org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
> at
> org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
> at
> org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
> ... 4 more
> INFO [STREAM-IN-/172
[jira] [Comment Edited] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761865#comment-13761865
]
Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:13 PM:
I have debugged Cassandra a bit and found the error occurs in the
*org.apache.cassandra.db.marshal.AbstractCompositeType*, which also can be seen
by the stack trace from the server log.
The problem is in the ByteBuffer passes to the *getWithShortLength* method:
There is no length contained in the buffer, only a field name for the actual
table. The local length variable will be constructed by using the two first
characters of the name. This will later provoke the *IllegalArgumentException*
when the position for the buffer is moved using the wrong calculated length.
My question is is it correct that the passed ByteBuffer does not contains a
length, or does the SSTableSimpleUnsortedWriter class mix up someting when it
creates it's SSTables?
was (Author: hbaastrup):
I have debugged Cassandra a bit and found the error occurs in the
org.apache.cassandra.db.marshal.AbstractCompositeType, which also can be seen
by the stack trace from the server log.
The problem is in the ByteBuffer passes to the getWithShortLength method: There
is no length contained in the buffer, only a field name for the actual table.
The local length variable will be constructed by using the two first characters
of the name. This will later provoke the IllegalArgumentException when the
position for the buffer is moved using the wrong calculated length.
My question is is it correct that the passed ByteBuffer does not contains a
length, or does the SSTableSimpleUnsortedWriter class mix up someting when it
creates it's SSTables?
> IllegalArgumentException when bulk loading
> --
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
>Reporter: Henrik Baastrup
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength
[jira] [Updated] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henrik Baastrup updated CASSANDRA-5987:
---
Description:
I have created SSTables with the
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
get:
WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
Streaming to the following hosts failed:
{quote}
[/127.0.0.1]
java.util.concurrent.ExecutionException:
org.apache.cassandra.streaming.StreamException: Stream failed
WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
error occurred
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
at java.lang.Thread.run(Unknown Source)
WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
{quote}
in the server log I have:
{quote}
ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
(line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
occurred
java.io.IOException: Too many retries for Header (cfId:
976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
transfer size: 54794676, compressed?: false)
at
org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
at
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
at
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
at
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
at
org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Unknown Source)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
at
org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
at
org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
at
org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
at
org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
at
org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
... 4 more
INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862
StreamResultFuture.java (line 180) [Stream
#7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864
StreamResultFuture.java (line 209) [Stream
#7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail
{quote}
was:
I have created SSTables with the
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
get:
WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
Streaming to the following hosts failed:
[/127.0.0.1]
java.util.concurrent.ExecutionException:
org.apache.cassandra.streaming.StreamException: Stream failed
WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed
ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
error occurred
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
at
org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHa
[jira] [Commented] (CASSANDRA-5987) IllegalArgumentException when bulk loading
[ https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761865#comment-13761865 ] Henrik Baastrup commented on CASSANDRA-5987: I have debugged Cassandra a bit and found the error occurs in the org.apache.cassandra.db.marshal.AbstractCompositeType, which also can be seen by the stack trace from the server log. The problem is in the ByteBuffer passes to the getWithShortLength method: There is no length contained in the buffer, only a field name for the actual table. The local length variable will be constructed by using the two first characters of the name. This will later provoke the IllegalArgumentException when the position for the buffer is moved using the wrong calculated length. My question is is it correct that the passed ByteBuffer does not contains a length, or does the SSTableSimpleUnsortedWriter class mix up someting when it creates it's SSTables? > IllegalArgumentException when bulk loading > -- > > Key: CASSANDRA-5987 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5987 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.0.0 on Linux >Reporter: Henrik Baastrup > > I have created SSTables with the > org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I > try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I > get: > WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream > failed > Streaming to the following hosts failed: > [/127.0.0.1] > java.util.concurrent.ExecutionException: > org.apache.cassandra.streaming.StreamException: Stream failed > WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream > failed > ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming > error occurred > java.io.IOException: Broken pipe > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > at sun.nio.ch.SocketDispatcher.write(Unknown Source) > at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) > at sun.nio.ch.IOUtil.write(Unknown Source) > at sun.nio.ch.SocketChannelImpl.write(Unknown Source) > at > org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43) > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384) > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372) > at java.lang.Thread.run(Unknown Source) > WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream > failed > in the server log I have: > ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java > (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error > occurred > java.io.IOException: Too many retries for Header (cfId: > 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, > transfer size: 54794676, compressed?: false) > at > org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522) > at > org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59) > at > org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45) > at > org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54) > at > org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287) > at java.lang.Thread.run(Unknown Source) > Caused by: java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Unknown Source) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130) > at > org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101) > at > org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254) > at > org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134) > at > org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88) > at > org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55) > ... 4 more > INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 > StreamResultFuture.java (line 180) [Stream > #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete > WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 > St
[jira] [Created] (CASSANDRA-5987) IllegalArgumentException when bulk loading
Henrik Baastrup created CASSANDRA-5987: -- Summary: IllegalArgumentException when bulk loading Key: CASSANDRA-5987 URL: https://issues.apache.org/jira/browse/CASSANDRA-5987 Project: Cassandra Issue Type: Bug Components: Core Environment: Cassandra 2.0.0 on Linux Reporter: Henrik Baastrup I have created SSTables with the org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I get: WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed Streaming to the following hosts failed: [/127.0.0.1] java.util.concurrent.ExecutionException: org.apache.cassandra.streaming.StreamException: Stream failed WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error occurred java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(Unknown Source) at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) at sun.nio.ch.IOUtil.write(Unknown Source) at sun.nio.ch.SocketChannelImpl.write(Unknown Source) at org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43) at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384) at org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372) at java.lang.Thread.run(Unknown Source) WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream failed in the server log I have: ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error occurred java.io.IOException: Too many retries for Header (cfId: 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208, transfer size: 54794676, compressed?: false) at org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522) at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59) at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45) at org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54) at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.IllegalArgumentException at java.nio.Buffer.limit(Unknown Source) at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55) at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64) at org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130) at org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101) at org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254) at org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134) at org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88) at org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55) ... 4 more INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862 StreamResultFuture.java (line 180) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864 StreamResultFuture.java (line 209) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
