[
https://issues.apache.org/jira/browse/CASSANDRA-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761882#comment-13761882
]
Henrik Baastrup edited comment on CASSANDRA-5987 at 9/9/13 2:30 PM:
--------------------------------------------------------------------
Attached my program there create the SSTables. Set-up the classpath to contain
all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name *nsmo_ds* with the following table:
{quote}
CREATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
{quote}
was (Author: hbaastrup):
Attached my program there create the SSTables. Set-up the classpath to
contain all necessary Cassandra jars and lunch the program like this:
java hba.cassandra.CsvToSSTables -p dns ./data/cdr_big.0.dns.sdf
Cassandra must contains a key space name nsmo_ds with the following table:
REATE TABLE nsmo_dns (
endtime bigint,
lastframetime bigint,
begintime bigint,
moduleid int,
callsequencenumber bigint,
partialcdanum int,
server_ip_ip0 bigint,
server_ip_ip1 bigint,
server_ip_ip2 bigint,
server_ip_ip3 bigint,
client_ip_ip0 bigint,
client_ip_ip1 bigint,
client_ip_ip2 bigint,
client_ip_ip3 bigint,
ip_src_ip0 bigint,
ip_src_ip1 bigint,
ip_src_ip2 bigint,
ip_src_ip3 bigint,
dnsenume164 varchar,
callduration bigint,
e164number varchar,
responsecode int,
internalcauses int,
queryname varchar,
querytype int,
queryclass int,
PRIMARY KEY (callsequencenumber,moduleid)
) ;
> IllegalArgumentException when bulk loading
> ------------------------------------------
>
> Key: CASSANDRA-5987
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5987
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.0.0 on Linux
> Reporter: Henrik Baastrup
> Attachments: CassandraLoader.tgz
>
>
> I have created SSTables with the
> org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter class, but when I
> try to load the SSTables with "sstableloader -d localhost nsmo_ds/nsmo_ds" I
> get:
> WARN 13:46:40,875 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> Streaming to the following hosts failed:
> {quote}
> [/127.0.0.1]
> java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> ERROR 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming
> error occurred
> java.io.IOException: Broken pipe
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:43)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:384)
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:372)
> at java.lang.Thread.run(Unknown Source)
> WARN 13:46:40,877 [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream
> failed
> {quote}
> in the server log I have:
> {quote}
> ERROR [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,859 StreamSession.java
> (line 409) [Stream #7d0193a0-1945-11e3-ac84-f9df5751357f] Streaming error
> occurred
> java.io.IOException: Too many retries for Header (cfId:
> 976cb58f-d1f4-39c6-b6cc-8aea7a64daca, #0, version: ja, estimated keys: 62208,
> transfer size: 54794676, compressed?: false)
> at
> org.apache.cassandra.streaming.StreamSession.doRetry(StreamSession.java:522)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:59)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:45)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:287)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.split(AbstractCompositeType.java:130)
> at
> org.apache.cassandra.io.sstable.ColumnNameHelper.minComponents(ColumnNameHelper.java:101)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.appendFromStream(SSTableWriter.java:254)
> at
> org.apache.cassandra.streaming.StreamReader.writeRow(StreamReader.java:134)
> at
> org.apache.cassandra.streaming.StreamReader.read(StreamReader.java:88)
> at
> org.apache.cassandra.streaming.messages.FileMessage$1.deserialize(FileMessage.java:55)
> ... 4 more
> INFO [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,862
> StreamResultFuture.java (line 180) [Stream
> #7d0193a0-1945-11e3-ac84-f9df5751357f] Session with /172.27.14.242 is complete
> WARN [STREAM-IN-/172.27.14.242] 2013-09-09 13:46:40,864
> StreamResultFuture.java (line 209) [Stream
> #7d0193a0-1945-11e3-ac84-f9df5751357f] Stream fail
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
