Re: Production Quality Ruby Driver?

[Theo Hultberg]

And cql-rb is full featured when it comes to CQL3. It supports all features
of Cassandra 1.2. For some of the Cassandra 2.0 features you have to wait
for a final version of 2.0, but the current prerelease is stable and well
tested.

yours
Theo


On Wed, Mar 19, 2014 at 5:21 PM, Theo Hultberg  wrote:

> I'm the author of cql-rb, the first one on your list. It runs in
> production in systems doing tens of thousands of operations per second.
> cequel is an ORM and its latest version runs on top of cql-rb.
>
> If you decide on using cql-rb I'm happy to help you out with any problems
> you might have, just open an issue on the GitHub project page.
>
> yours
> Theo
>
>
> On Mon, Mar 17, 2014 at 6:55 PM, NORD SC wrote:
>
>> Hi,
>>
>> I am looking for a Ruby driver that is production ready and truly
>> supports CQL 3. Can anyone strongly recommend one in particular?
>>
>>
>> I found
>>
>> - https://github.com/iconara/cql-rb
>> - https://github.com/kreynolds/cassandra-cql
>> - https://github.com/cequel/cequel
>>
>>
>> Jan
>>
>>
>

Re: Production Quality Ruby Driver?

[Theo Hultberg]

I'm the author of cql-rb, the first one on your list. It runs in production
in systems doing tens of thousands of operations per second. cequel is an
ORM and its latest version runs on top of cql-rb.

If you decide on using cql-rb I'm happy to help you out with any problems
you might have, just open an issue on the GitHub project page.

yours
Theo


On Mon, Mar 17, 2014 at 6:55 PM, NORD SC  wrote:

> Hi,
>
> I am looking for a Ruby driver that is production ready and truly supports
> CQL 3. Can anyone strongly recommend one in particular?
>
>
> I found
>
> - https://github.com/iconara/cql-rb
> - https://github.com/kreynolds/cassandra-cql
> - https://github.com/cequel/cequel
>
>
> Jan
>
>

Re: Problems with adding datacenter and schema version disagreement

[olek.stas...@gmail.com]

Bump, could anyone comment this behaviour, is it correct, or should I
create Jira task for this problems?
regards
Olek

2014-03-18 16:49 GMT+01:00 olek.stas...@gmail.com :
> Oh, one more question: what should be configuration for storing
> system_traces keyspace? Should it be replicated or stored locally?
> Regards
> Olek
>
> 2014-03-18 16:47 GMT+01:00 olek.stas...@gmail.com :
>> Ok, i've dropped all system keyspaces, rebuild cluster and recover
>> schema, now everything looks ok.
>> But main goal of operations was to add new datacenter to cluster.
>> After starting node in new cluster two schema versions appear, one
>> version is held by 6 nodes of first datacenter, second one is in newly
>> added node in new datacenter. Sth like this:
>> nodetool status
>> Datacenter: datacenter1
>> ===
>> Status=Up/Down
>> |/ State=Normal/Leaving/Joining/Moving
>> --  AddressLoad   Tokens  Owns   Host ID
>> Rack
>> UN  192.168.1.1  50.19 GB   1   0,5%
>> c9323f38-d9c4-4a69-96e3-76cd4e1a204e  rack1
>> UN  192.168.1.2  54.83 GB   1   0,3%
>> ad1de2a9-2149-4f4a-aec6-5087d9d3acbb  rack1
>> UN  192.168.1.3  51.14 GB   1   0,6%
>> 0ceef523-93fe-4684-ba4b-4383106fe3d1  rack1
>> UN  192.168.1.4  54.31 GB   1   0,7%
>> 39d15471-456d-44da-bdc8-221f3c212c78  rack1
>> UN  192.168.1.5  53.36 GB   1   0,3%
>> 7fed25a5-e018-43df-b234-47c2f118879b  rack1
>> UN  192.168.1.6  39.89 GB   1   0,1%
>> 9f54fad6-949a-4fa9-80da-87efd62f3260  rack1
>> Datacenter: DC1
>> ===
>> Status=Up/Down
>> |/ State=Normal/Leaving/Joining/Moving
>> --  AddressLoad   Tokens  Owns   Host ID
>> Rack
>> UN  192.168.1.7  100.77 KB  256 97,4%
>> ddb1f913-d075-4840-9665-3ba64eda0558  RAC1
>>
>> describe cluster;
>> Cluster Information:
>>Name: Metadata Cluster
>>Snitch: org.apache.cassandra.locator.GossipingPropertyFileSnitch
>>Partitioner: org.apache.cassandra.dht.RandomPartitioner
>>Schema versions:
>> 8fe34841-4f2a-3c05-97f2-15dd413d71dc: [192.168.1.7]
>>
>> 4ad381b6-df5a-3cbc-ba5a-0234b74d2383: [192.168.1.1, 192.168.1.2,
>> 192.168.1.3, 192.168.1.4, 192.168.1.5, 192.168.1.6]
>>
>> All keyspaces are now configured to keep data in datacenter1.
>> I assume, that It's not correct behaviour, is it true?
>> Could you help me, how can I safely add new DC to the cluster?
>>
>> Regards
>> Aleksander
>>
>>
>> 2014-03-14 18:28 GMT+01:00 olek.stas...@gmail.com :
>>> Ok, I'll do this during the weekend, I'll give you a feedback on Monday.
>>> Regards
>>> Aleksander
>>>
>>> 14 mar 2014 18:15 "Robert Coli"  napisał(a):
>>>
 On Fri, Mar 14, 2014 at 12:40 AM, olek.stas...@gmail.com
  wrote:
>
> OK, I see, so the data files stay in place, i have to just stop
> cassandra on whole cluster, remove system schema and then start
> cluster and recreate all keyspaces with all column families? Data will
> be than loaded automatically from existing ssstables, right?


 Right. If you have clients reading while loading the schema, they may get
 exceptions.

>
> So one more question: what about KS system_traces? should it be
> removed and recreted? What data it's holding?


 It's holding data about tracing, a profiling feature. It's safe to nuke.

 =Rob

Re: Gossip intermittently marks node as DOWN

[Phil Luckhurst]

I think we've found the issue!

It seems that the times on those Cassandra servers was being kept in sync by
vmware tools using the time of the vmware host machine. We have now turned
that off and are using the ntp service to keep the times in sync like we do
for our physical servers and we have not seen the gossip failures for the
last 24 hours.

--
Phil



--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Gossip-intermittently-marks-node-as-DOWN-tp7593189p7593569.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Issue retrieving blobs with CQL

[Andrew Cobley]

Apologies,

this seems to be addressed in the thread :

https://groups.google.com/a/lists.datastax.com/forum/#!searchin/java-driver-user/blob$20ByteBuffer/java-driver-user/4_KegVX0teo/2OOZ8YOwtBcJ

Andy

On 19 Mar 2014, at 11:55, Andrew Cobley 
mailto:a.e.cob...@dundee.ac.uk>> wrote:

Cassandra 2.0.4

I’m using blobs to save and retrieve images in cassanda.   However the  blob I 
get back is not the blob I saved !

I’m saving to CQL like this:

byte[] b= {(byte)10,(byte)20,(byte)30};
int length=b.length;
ByteBuffer buffer =ByteBuffer.wrap(b);
Session session = cluster.connect("keyspace2");
PreparedStatement ps = session.prepare("insert into tweets ( image, user, 
interaction_time,imagelength) values(?,?,?,?)");
BoundStatement boundStatement = new BoundStatement(ps);
session.execute( boundStatement.bind( buffer, "majed",  
convertor.getTimeUUID(),length));

and retrieving like this:

Session session = cluster.connect("keyspace2");
PreparedStatement ps = session.prepare("select image,user,imagelength from 
tweets where user =?");
BoundStatement boundStatement = new BoundStatement(ps);
ResultSet rs =session.execute(boundStatement.bind( "majed"));
ByteBuffer bImage=null;
for (Row row : rs) {
   bImage = row.getBytes("image") ;
   System.out.println ("Image ");
   for (int i=0; i< 40;i++){
  System.out.print(String.format("%x ",bImage.get(i)));
   if ((i+1)%16 ==0) System.out.println();
   }
   System.out.println();
}
session.close();

So, I should be saving byte array of 3 long, and the  buffer in the save 
section is correct.  However when I run the retrieve section I’m getting back 
the following:

82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 3 a 14 1e

You can see the saved bytes are the last three and there is extra padding at 
the front.

Worse if I change the Select statement to:
PreparedStatement ps = session.prepare("select user,image,imagelength from 
tweets where user =?”);

I get back:

82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 5 6d 61 6a 65
64 0 0 0 3 a 14 1e

If you look you can see the username “majed” in the returned blob (6d 61 6a 65 
64).

any ideas whats going on here ?

Many thanks

Andy

The University of Dundee is a registered Scottish Charity, No: SC015096


The University of Dundee is a registered Scottish Charity, No: SC015096

Upgrade 1.2.11 to 2.0.6: some errors

[Nicolas Lalevée]

Hi,

On our test cluster, we tried a upgrade of Cassandra from 1.22.1 to 2.0.6. It 
was not straight forward so I would like to know if it is expected, so I can do 
it safely on prod.

The first time we tried, the first upgrading node refused to start with this 
error:

ERROR [main] 2014-03-19 10:50:31,363 CassandraDaemon.java (line 488) Exception 
encountered during startup
java.lang.RuntimeException: Incompatible SSTable found.  Current version jb is 
unable to read file: /var/lib/cassandra/d
ata/system/NodeIdInfo/system-NodeIdInfo-hf-4.  Please run upgradesstables.
at 
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:415)
at 
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:392)
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:309)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:266)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
at 
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:514)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:237)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:471)
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:560)

I've read again the NEWS.txt [1], and as far as I understand, upgradesstables 
is only required for < 1.2.9. But maybe I don't understand correctly the 
paragraph:
- Upgrading is ONLY supported from Cassandra 1.2.9 or later. This
  goes for sstable compatibility as well as network.  When
  upgrading from an earlier release, upgrade to 1.2.9 first and
  run upgradesstables before proceeding to 2.0.

So we did the required upgradesstables. The node started successfully.

I have checked on our prod cluster, there is also some hf files, on all nodes, 
all being /var/lib/cassandra/data/system/Versions/system-Versions-hf-*
And I have tried many upgradesstables command, there are still lying there.
# nodetool upgradesstables system Versions
Exception in thread "main" java.lang.IllegalArgumentException: Unknown table/cf 
pair (system.Versions)
# nodetool upgradesstables system
# nodetool upgradesstables
# nodetool upgradesstables -a system
# ls /var/lib/cassandra/data/system/Versions/*-hf-* | wc -l
15

I did not try "nodetool upgradesstables -a" since we have a lot of data.

I guess this will cause me trouble if I try to upgrade in prod ? Is there a bug 
I should report ?

Continuing on our test cluster, we upgraded the second node. And during the 
time we were running with 2 different versions of cassandra, there was errors 
in the logs:

ERROR [WRITE-/10.10.0.41] 2014-03-19 11:23:27,523 OutboundTcpConnection.java 
(line 234) error writing to /10.10.0.41
java.lang.RuntimeException: Cannot convert filter to old super column format. 
Update all nodes to Cassandra 2.0 first.
at 
org.apache.cassandra.db.SuperColumns.sliceFilterToSC(SuperColumns.java:357)
at 
org.apache.cassandra.db.SuperColumns.filterToSC(SuperColumns.java:258)
at 
org.apache.cassandra.db.ReadCommandSerializer.serializedSize(ReadCommand.java:192)
at 
org.apache.cassandra.db.ReadCommandSerializer.serializedSize(ReadCommand.java:134)
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
at 
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:251)
at 
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:203)
at 
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:151)

I confirm we do have old style super columns which were designed when cassandra 
was 1.0.x. Since in our test cluster the replication factor is 1, I can see 
errors on the client side, since 1 node among 2 was down. So I don't know for 
sure if this error in cassandra affected the client, the time frame is too 
short to be sure from the logs. In prod we have a replication factor of 3. If 
we'll do a such upgrade in prod, node by node to avoid any downtime, will the 
client still see write errors during the time there will be mixed versions of 
cassandra ?

Nicolas

[1] 
https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/cassandra-2.0.6

Issue retrieving blobs with CQL

[Andrew Cobley]

Cassandra 2.0.4

I’m using blobs to save and retrieve images in cassanda.   However the  blob I 
get back is not the blob I saved !

I’m saving to CQL like this:

byte[] b= {(byte)10,(byte)20,(byte)30};
int length=b.length;
ByteBuffer buffer =ByteBuffer.wrap(b);
Session session = cluster.connect("keyspace2");
PreparedStatement ps = session.prepare("insert into tweets ( image, user, 
interaction_time,imagelength) values(?,?,?,?)");
BoundStatement boundStatement = new BoundStatement(ps);
session.execute( boundStatement.bind( buffer, "majed",  
convertor.getTimeUUID(),length));

and retrieving like this:

Session session = cluster.connect("keyspace2");
PreparedStatement ps = session.prepare("select image,user,imagelength from 
tweets where user =?");
BoundStatement boundStatement = new BoundStatement(ps);
ResultSet rs =session.execute(boundStatement.bind( "majed"));
ByteBuffer bImage=null;
for (Row row : rs) {
   bImage = row.getBytes("image") ;
   System.out.println ("Image ");
   for (int i=0; i< 40;i++){
  System.out.print(String.format("%x ",bImage.get(i)));
   if ((i+1)%16 ==0) System.out.println();
   }
   System.out.println();
}
session.close();

So, I should be saving byte array of 3 long, and the  buffer in the save 
section is correct.  However when I run the retrieve section I’m getting back 
the following:

82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 3 a 14 1e

You can see the saved bytes are the last three and there is extra padding at 
the front.

Worse if I change the Select statement to:
PreparedStatement ps = session.prepare("select user,image,imagelength from 
tweets where user =?”);

I get back:

82 0 0 8 0 0 0 28 0 0 0 2 0 0 0 5
0 0 0 3 0 0 0 1 0 0 0 5 6d 61 6a 65
64 0 0 0 3 a 14 1e

If you look you can see the username “majed” in the returned blob (6d 61 6a 65 
64).

any ideas whats going on here ?

Many thanks

Andy

The University of Dundee is a registered Scottish Charity, No: SC015096

Re: Setting up Cassandra across private network

[Marcin Cabaj]

Hi,

you should set 'listen_address' to ip address of eth4.
'listen_address' is for communication between cassandra nodes (not clients).

Btw I hope you don't use NFS for commit log directory.

-- 
cheers
mc


On Wed, Mar 19, 2014 at 5:14 AM, Le Xu  wrote:

> I'm currently using Cassandra 1.23 and I'm trying to set up a cluster on
> the private network,  so that nodes can communicate through eth4 inet
> address instead of eth0 inet address.
> My current yaml file specify eth4 address in the seed field. However the
> the rpc address is set to 0.0.0.0. Both listen address and broadcast
> address are not set.
>
> Then I got error message:
>
> ERROR 22:52:21,936 Exception encountered during startup
> java.lang.RuntimeException: No other nodes seen!  Unable to bootstrap.If
> you intended to start a single-node cluster, you should make sure your
> broadcast_address (or listen_address) is listed as a seed.  Otherwise, you
> need to determine why the seed being contacted has no knowledge of the rest
> of the cluster.  Usually, this can be solved by giving all nodes the same
> seed list.
>
> My guess is the nodes were not be able to communicate with each other but
> some unspecified field is resolved to eth0 address. Could it be the problem
> with broadcast/listen address? My nodes share filesystem through NFS so if
> eth4 address needs to be specified receptively on each node then separate
> config file will be necessary.
>

Re: How to extract information from commit log?

[Artur Kronenberg]

Hi,

we did something similar. We did utilize some cassandra code though and 
wrote a custom commitlog reader that outputs our data into a readable form.


You can look here: 
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/1.1.9/org/apache/cassandra/db/commitlog/CommitLogReplayer.java


This code is used to replay commitlogs when starting up cassandra. It 
has the ability to deserialize and transform the data into what you'll need.


-- artur


On 18/03/14 19:32, Han,Meng wrote:


Hi Jonathan,

Thank you for the timely reply. I am doing this experiment on a 
continuous basis. To be more specific, I will issue a large amount of 
read and write operations to a particular key in a short time 
interval. I'd like to know the order that write operations happens at 
each replica. TImestamps definitely help to determine order, but the 
WRITETIME and SStable2Json both looks me only return the timestamps 
when that key was updated the moment the WRITETIME/SStable2Json is 
issued. It looks like a one time thing to me. Or put in another way, 
if I want to get the write time for all write operations in that short 
invertal to determine a total order for write on that replia I have to 
constantly issue WRITETIME to this replica?  Correct me if I am wrong 
here.


Light me up pleease!

On Tue, 18 Mar 2014 15:05:07 -0400, Jonathan Lacefield wrote:


Hello,
  Is this a one time investigative item or are you looking to set 
something up to do this continuously?  Don't recommend trying to read 
the commit log.
  You can always use the WRITETIME function in CQL or look within 
SSTables via the SStable2Json utility to see write times for 
particular versions of partitions.

Jonathan

Jonathan Lacefield
Solutions Architect, DataStax
(404) 822 3487




On Tue, Mar 18, 2014 at 2:25 PM, Han,Meng > wrote:


Hi Cassandra hackers!

I have a question regarding extracting useful information from
commit log.

Since its a binary log, how should I extract information such as
timestamp, values from it? Does anyone know any binary log reader
that I can use directly to read commit log?
If there is no such reader, could someone give me some advice hwo
I can wrote such a reader?

Particularly, I want to know the order that write operations
happens at each replica(cassandra server node) along with their
timestamps, Does anyone know other methods how I can get this
information without instrumenting Cassandra code?

Any help is appreciated!

Cheers,
Meng