Re: Compaction issues, 2.0.12
Thanks Rob, Jeff. I have updated the Jira issue with my information. On 6 July 2015 at 23:46, Jeff Ferland wrote: > I’ve seen the same thing: > https://issues.apache.org/jira/browse/CASSANDRA-9577 > > I’ve had cases where a restart clears the old tables, and I’ve had cases > where a restart considers the old tables to be live. > > On Jul 6, 2015, at 1:51 PM, Robert Coli wrote: > > On Mon, Jul 6, 2015 at 1:46 PM, Jeff Williams > wrote: > >> 1) Cassandra version is 2.0.12. >> > > >> 2) Interesting. Looking at JMX org.apache.cassandra.db -> ColumnFamilies >> -> trackcontent -> track_content -> Attributes, I get: >> >> LiveDiskSpaceUsed: 17788740448, i.e. ~17GB >> LiveSSTableCount: 3 >> TotalDiskSpaceUsed: 55714084629, i.e. ~55GB >> >> So it obviously knows about the extra disk space even though the "live" >> space looks correct. I couldn't find anything to identify the actual files >> though. >> > > That's what I would expect. > > >> 3) So that was even more interesting. After restarting the cassandra >> daemon, the sstables were not deleted and now the same JMX attributes are: >> >> LiveDiskSpaceUsed: 55681040579, i.e. ~55GB >> LiveSSTableCount: 8 >> TotalDiskSpaceUsed: 55681040579, i.e. ~55GB >> >> So some of my non-live tables are back live again, and obviously some of >> the big ones!! >> > > This is permanently fatal to consistency; sorry if I was not clear enough > that if they were not live, there was some risk of Cassandra considering > them live again upon restart. > > If I were you, I would either stop the node and remove the files you know > shouldn't be live or do a major compaction ASAP. > > The behavior you just encountered sounds like a bug, and it is a rather > serious one. SSTables which should be dead being marked live is very bad > for consistency. > > Do you see any exceptions in your logs or anything? If you can repro, you > should file a JIRA ticket with the apache project... > > =Rob > > >
Re: Wrong peers
Anton, I have also seen this issue with decommissioned nodes remaining in the system.peers table. On the bright side, they can be safely removed from the system.peers table without issue. You will have to check every node in the cluster since this is a local setting per node. Jeff On 6 July 2015 at 22:45, nowarry wrote: > Hey guys, > > I'm using Ruby driver( http://datastax.github.io/ruby-driver/ ) for > backup scripts. I tried to discover all peers and got wrong peers that are > different with nodetool status. > > = > Status=Up/Down > |/ State=Normal/Leaving/Joining/Moving > -- Address Load Tokens OwnsHost ID > Rack > UN 10.40.231.53 1.18 TB256 ? > b2d877d7-f031-4190-8569-976bb0ce034f RACK01 > UN 10.40.231.11 1.24 TB256 ? > e15cda1c-65cc-40cb-b85c-c4bd665d02d7 RACK01 > > cqlsh> use system; > cqlsh:system> select peer from system.peers; > > peer > -- > 10.40.231.31 > 10.40.231.53 > > (2 rows) > > What to do with these old peers, whether they can be removed without > consequences since they are not in production cluster? And how to keep up > to date the peers? > > -- > Anton Koshevoy > >
Re: Compaction issues, 2.0.12
Thanks Rob. 1) Cassandra version is 2.0.12. 2) Interesting. Looking at JMX org.apache.cassandra.db -> ColumnFamilies -> trackcontent -> track_content -> Attributes, I get: LiveDiskSpaceUsed: 17788740448, i.e. ~17GB LiveSSTableCount: 3 TotalDiskSpaceUsed: 55714084629, i.e. ~55GB So it obviously knows about the extra disk space even though the "live" space looks correct. I couldn't find anything to identify the actual files though. 3) So that was even more interesting. After restarting the cassandra daemon, the sstables were not deleted and now the same JMX attributes are: LiveDiskSpaceUsed: 55681040579, i.e. ~55GB LiveSSTableCount: 8 TotalDiskSpaceUsed: 55681040579, i.e. ~55GB So some of my non-live tables are back live again, and obviously some of the big ones!! Jef On 6 July 2015 at 20:26, Robert Coli wrote: > On Mon, Jul 6, 2015 at 7:25 AM, Jeff Williams > wrote: > >> So it appears that the 10 sstables that were compacted >> to >> /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57372 >> are still sitting around, with a couple of them being very large! >> Any ideas for what I can check? Can I just delete the old sstables? (I'm >> guessing not). >> > > 1) What version of Cassandra? > > 2) There are JMX methods in some versions to identify which files are live > in the data dir, you could try to use those to see if those files are live. > Does "load" seem to show that they are live from the perspective of > Cassandra? > > 3) If they are not live, restarting the Cassandra daemon on the node will > likely result in them being deleted. > > =Rob > >
Compaction issues, 2.0.12
Hi,
I have a keyspace which is using about 19GB on most nodes in the cluster.
However on one node the data directory for the only table in that keyspace
now uses 52GB. It seems that old sstables are not removed after compaction.
Here is a view of the data directory Data files before compaction (I didn't
think to check sizes at the time):
$ ls /var/lib/cassandra/data/trackcontent/track_content/*Data.db
trackcontent-track_content-jb-30852-Data.db
trackcontent-track_content-jb-30866-Data.db
trackcontent-track_content-jb-37136-Data.db
trackcontent-track_content-jb-43371-Data.db
trackcontent-track_content-jb-44676-Data.db
trackcontent-track_content-jb-51845-Data.db
trackcontent-track_content-jb-55339-Data.db
trackcontent-track_content-jb-57353-Data.db
trackcontent-track_content-jb-57366-Data.db
trackcontent-track_content-jb-57367-Data.db
trackcontent-track_content-jb-57368-Data.db
Then when compaction was run I got the following in the logs:
INFO [CompactionExecutor:3546] 2015-07-06 12:01:04,246 CompactionTask.java
(line 120) Compacting [
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30852-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30866-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-37136-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-43371-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-44676-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-51845-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-55339-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57353-Data.db')
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57366-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57371-Data.db'),
]
INFO [CompactionExecutor:3546] 2015-07-06 13:34:31,458 CompactionTask.java
(line 296) Compacted 10 sstables to
[/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57372,].
36,182,766,725 bytes to 16,916,868,412 (~46% of original) in 5,607,211ms =
2.877221MB/s. 84,600,535 total partitions merged to 40,835,912. Partition
merge counts were {1:864528, 2:36318841, 3:3523852, 4:124091, 5:6051, 6:25,
}
But after compaction completes, the data directory still looks like:
$ ls -l /var/lib/cassandra/data/trackcontent/track_content/*Data.db
-rw-r--r-- 1 cassandra cassandra 16642164891 Jun 29 06:55
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30852-Data.db
-rw-r--r-- 1 cassandra cassandra 17216513377 Jun 30 08:36
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30866-Data.db
-rw-r--r-- 1 cassandra cassandra 813683923 Jun 30 12:03
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-37136-Data.db
-rw-r--r-- 1 cassandra cassandra 855070477 Jun 30 13:15
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-43371-Data.db
-rw-r--r-- 1 cassandra cassandra 209921645 Jun 30 13:28
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-44676-Data.db
-rw-r--r-- 1 cassandra cassandra 213532360 Jul 2 03:16
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-51845-Data.db
-rw-r--r-- 1 cassandra cassandra14763933 Jul 2 07:58
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-52921-Data.db
-rw-r--r-- 1 cassandra cassandra16955005 Jul 2 08:09
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-53037-Data.db
-rw-r--r-- 1 cassandra cassandra61322156 Jul 2 23:34
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-55339-Data.db
-rw-r--r-- 1 cassandra cassandra61898096 Jul 4 01:02
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57353-Data.db
-rw-r--r-- 1 cassandra cassandra75912668 Jul 5 21:23
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57366-Data.db
-rw-r--r-- 1 cassandra cassandra32747132 Jul 6 11:01
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57371-Data.db
-rw-r--r-- 1 cassandra cassandra 16916868412 Jul 6 13:34
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57372-Data.db
-rw-r--r-- 1 cassandra cassandra 8392714 Jul 6 13:24
/var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57373-Data.db
So it appears that the 10 sstables that were c
Re: Decommissioned node still in Gossip
Solved. Finally found that there was one node with 172.29.8.8 in gossipinfo and appearing incomplete in status: Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack UL 172.29.8.8 13.28 GB 256 9.5% null 1 Restarting this node removed the decommissioned node from gossip. Jeff On 3 July 2015 at 13:09, Jeff Williams wrote: > I'm thinking I'll have to find something similar for 2.0. I just don't > understand where the node is coming from! > > Jeff > > On 1 July 2015 at 10:21, Vitalii Skakun wrote: > >> Hi, >> >> just a guess, there was a possibility to purge gossip state on a node, at >> least in version 1.2 >> >> http://docs.datastax.com/en/cassandra/1.2/cassandra/architecture/architectureGossipPurge_t.html >> >> the trick was to add -Dcassandra.load_ring_state=false somehow to the >> jvm parameters >> >> I'm not sure if it works for 2.0 (since it's not in the doc for 2.x) >> >> 2015-07-01 10:23 GMT+03:00 Jeff Williams : >> >>> Thanks for the tip Aiman, but this node is not in the seed list anywhere. >>> >>> Jeff >>> >>> On 30 June 2015 at 18:16, Aiman Parvaiz wrote: >>> >>>> I was having exactly the same issue with the same version, check your >>>> seed list and make sure it contains only the live nodes, I know that seeds >>>> are only read when cassandra starts but updating the seed list to live >>>> nodes and then doing a roiling restart fixed this issue for me. >>>> I hope this helps you. >>>> >>>> Thanks >>>> >>>> Sent from my iPhone >>>> >>>> On Jun 30, 2015, at 4:42 AM, Jeff Williams >>>> wrote: >>>> >>>> Hi, >>>> >>>> I have a cluster which had 4 datacenters running 2.0.12. Last week one >>>> of the datacenters was decommissioned using nodetool decommission on each >>>> of the servers in turn. This seemed to work fine until one of the nodes >>>> started appearing in the logs of all of the remaining servers with messages >>>> like: >>>> >>>> INFO [GossipStage:3] 2015-06-30 11:22:39,189 Gossiper.java (line 924) >>>> InetAddress /172.29.8.8 is now DOWN >>>> INFO [GossipStage:3] 2015-06-30 11:22:39,190 StorageService.java (line >>>> 1773) Removing tokens [...] for /172.29.8.8 >>>> >>>> These come up in the log every minute or two. I believe it may have >>>> re-appeared after a repair, but I'm not sure. >>>> >>>> The problem is that this node does not exist in "nodetool status", >>>> "nodetool gossipinfo" or in the system.peers table. So how can tell the >>>> cluster that this node is decommissioned? >>>> >>>> Regards, >>>> Jeff >>>> >>>> >>> >> >> >> -- >> >> >> with best regards, >> Vitalii Skakun >> > >
Re: Decommissioned node still in Gossip
I'm thinking I'll have to find something similar for 2.0. I just don't understand where the node is coming from! Jeff On 1 July 2015 at 10:21, Vitalii Skakun wrote: > Hi, > > just a guess, there was a possibility to purge gossip state on a node, at > least in version 1.2 > > http://docs.datastax.com/en/cassandra/1.2/cassandra/architecture/architectureGossipPurge_t.html > > the trick was to add -Dcassandra.load_ring_state=false somehow to the jvm > parameters > > I'm not sure if it works for 2.0 (since it's not in the doc for 2.x) > > 2015-07-01 10:23 GMT+03:00 Jeff Williams : > >> Thanks for the tip Aiman, but this node is not in the seed list anywhere. >> >> Jeff >> >> On 30 June 2015 at 18:16, Aiman Parvaiz wrote: >> >>> I was having exactly the same issue with the same version, check your >>> seed list and make sure it contains only the live nodes, I know that seeds >>> are only read when cassandra starts but updating the seed list to live >>> nodes and then doing a roiling restart fixed this issue for me. >>> I hope this helps you. >>> >>> Thanks >>> >>> Sent from my iPhone >>> >>> On Jun 30, 2015, at 4:42 AM, Jeff Williams >>> wrote: >>> >>> Hi, >>> >>> I have a cluster which had 4 datacenters running 2.0.12. Last week one >>> of the datacenters was decommissioned using nodetool decommission on each >>> of the servers in turn. This seemed to work fine until one of the nodes >>> started appearing in the logs of all of the remaining servers with messages >>> like: >>> >>> INFO [GossipStage:3] 2015-06-30 11:22:39,189 Gossiper.java (line 924) >>> InetAddress /172.29.8.8 is now DOWN >>> INFO [GossipStage:3] 2015-06-30 11:22:39,190 StorageService.java (line >>> 1773) Removing tokens [...] for /172.29.8.8 >>> >>> These come up in the log every minute or two. I believe it may have >>> re-appeared after a repair, but I'm not sure. >>> >>> The problem is that this node does not exist in "nodetool status", >>> "nodetool gossipinfo" or in the system.peers table. So how can tell the >>> cluster that this node is decommissioned? >>> >>> Regards, >>> Jeff >>> >>> >> > > > -- > > > with best regards, > Vitalii Skakun >
Re: Decommissioned node still in Gossip
Thanks for the tip Aiman, but this node is not in the seed list anywhere. Jeff On 30 June 2015 at 18:16, Aiman Parvaiz wrote: > I was having exactly the same issue with the same version, check your seed > list and make sure it contains only the live nodes, I know that seeds are > only read when cassandra starts but updating the seed list to live nodes > and then doing a roiling restart fixed this issue for me. > I hope this helps you. > > Thanks > > Sent from my iPhone > > On Jun 30, 2015, at 4:42 AM, Jeff Williams > wrote: > > Hi, > > I have a cluster which had 4 datacenters running 2.0.12. Last week one of > the datacenters was decommissioned using nodetool decommission on each of > the servers in turn. This seemed to work fine until one of the nodes > started appearing in the logs of all of the remaining servers with messages > like: > > INFO [GossipStage:3] 2015-06-30 11:22:39,189 Gossiper.java (line 924) > InetAddress /172.29.8.8 is now DOWN > INFO [GossipStage:3] 2015-06-30 11:22:39,190 StorageService.java (line > 1773) Removing tokens [...] for /172.29.8.8 > > These come up in the log every minute or two. I believe it may have > re-appeared after a repair, but I'm not sure. > > The problem is that this node does not exist in "nodetool status", > "nodetool gossipinfo" or in the system.peers table. So how can tell the > cluster that this node is decommissioned? > > Regards, > Jeff > >
Decommissioned node still in Gossip
Hi, I have a cluster which had 4 datacenters running 2.0.12. Last week one of the datacenters was decommissioned using nodetool decommission on each of the servers in turn. This seemed to work fine until one of the nodes started appearing in the logs of all of the remaining servers with messages like: INFO [GossipStage:3] 2015-06-30 11:22:39,189 Gossiper.java (line 924) InetAddress /172.29.8.8 is now DOWN INFO [GossipStage:3] 2015-06-30 11:22:39,190 StorageService.java (line 1773) Removing tokens [...] for /172.29.8.8 These come up in the log every minute or two. I believe it may have re-appeared after a repair, but I'm not sure. The problem is that this node does not exist in "nodetool status", "nodetool gossipinfo" or in the system.peers table. So how can tell the cluster that this node is decommissioned? Regards, Jeff
Find rows without a column
Hi, It doesn't look like this is possible, but can I select all rows missing a certain column? The equivalent of "select * where col is null" in SQL. Regards, Jeff
Re: cqlsh commands
You should be able to get most of this information from the datastax documentation, e.g. http://www.datastax.com/docs/1.0/references/cql/index for version 1.0. Jeff On Jun 20, 2012, at 1:47 PM, Thierry Templier wrote: > Hello, > > I'm using cqlsh tool to create column families and it works fine. However I > don't know how to use the following commands: > > Documented commands (type help ): > > ASSUME CAPTURE DESC DESCRIBE EXIT HELP SELECT SHOW SOURCE USE > > Moreover I can't display the related documentations using the help command. > > Thanks very much for your help. > Thierry
Re: Confusion regarding the terms "replica" and "replication factor"
On May 30, 2012, at 10:32 PM, Edward Capriolo wrote: > > The snitch does use the datacenter and the rack but almost all > deployments use a single rack per DC, because when you have more then > one rack in a data center the NTS snitch has some logic to spread the > data between racks. (most people do not want this behavior) > Out of curiosity, why would most people not want this behaviour? It seems like a good idea from a availability perspective. Jeff
Re: Confusion regarding the terms "replica" and "replication factor"
First, note that replication is done at the row level, not at the node level.
That line should look more like:
placement_strategy = 'NetworkTopologyStrategy' and strategy_options = {DC1:
1,DC2: 1,DC3: 1 }
This means that each row will have one copy in each DC and within each DC it's
placement will be according to the partitioner, so could be on any of the nodes
in the each DC.
So, don't think of it as nodes replicating, but rather as how nodes should
store a copy of each row in each DC.
Also, replication does not relate the the seed nodes. Seed nodes allow the
nodes to find each other initially, but are not special otherwise - any node
can be used as a seed node.
So if you had a strategy like:
placement_strategy = 'NetworkTopologyStrategy' and strategy_options = {DC1:
3,DC2: 2,DC3: 1 }
Each row would exist on 3 of 4 nodes in DC1, 2 of 4 nodes in DC2 and on one of
the nodes in DC3. Again, with the placement in each DC due to the partitioner,
based on the row key.
Jeff
On May 29, 2012, at 11:25 PM, David Fischer wrote:
> Ok now i am confused :),
>
> ok if i have the following
> placement_strategy = 'NetworkTopologyStrategy' and strategy_options =
> {DC1:R1,DC2:R1,DC3:R1 }
>
> this means in each of my datacenters i will have one full replica that
> also can be seed node?
> if i have 3 node in addition to the DC replica's with normal token
> calculations a key can be in any datacenter plus on each of the
> replicas right?
> It will show 12 nodes total in its ring
>
> On Thu, May 24, 2012 at 2:39 AM, aaron morton wrote:
>> This is partly historical. NTS (as it is now) has not always existed and was
>> not always the default. In days gone by used to be a fella could run a
>> mighty fine key-value store using just a Simple Replication Strategy.
>>
>> A different way to visualise it is a single ring with a Z axis for the DC's.
>> When you look at the ring from the top you can see all the nodes. When you
>> look at it from the side you can see the nodes are on levels that correspond
>> to their DC. Simple Strategy looks at the ring from the top. NTS works
>> through the layers of the ring.
>>
>>> If the hierarchy is Cluster ->
>>> DataCenter -> Node, why exactly do we need globally unique node tokens
>>> even though nodes are at the lowest level in the hierarchy.
>> Nodes having a DC is a feature of *some* snitches and utilised by the *some*
>> of the replication strategies (and by the messaging system for network
>> efficiency). For background, mapping from row tokens to nodes is based on
>> http://en.wikipedia.org/wiki/Consistent_hashing
>>
>> Hope that helps.
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 24/05/2012, at 1:07 AM, java jalwa wrote:
>>
>>> Thanks Aaron. That makes things clear.
>>> So I guess the 0 - 2^127 range for tokens corresponds to a cluster
>>> -level top-level ring. and then you add some logic on top of that with
>>> NTS to logically segment that range into sub-rings as per the notion
>>> of data clusters defined in NTS. Whats the advantage of having a
>>> single top-level ring ? intuitively it seems like each replication
>>> group could have a separate ring so that the same tokens can be
>>> assigned to nodes in different DC. If the hierarchy is Cluster ->
>>> DataCenter -> Node, why exactly do we need globally unique node tokens
>>> even though nodes are at the lowest level in the hierarchy.
>>>
>>> Thanks again.
>>>
>>>
>>> On Wed, May 23, 2012 at 3:14 AM, aaron morton
>>> wrote:
> Now if a row key hash is mapped to a range owned by a node in DC3,
> will the Node in DC3 still store the key as determined by the
> partitioner and then walk the ring and store 2 replicas each in DC1
> and DC2 ?
No, only nodes in the DC's specified in the NTS configuration will be
replicas.
> Or will the co-ordinator node be aware of the
> replica placement strategy,
> and override the partitioner's decision and walk the ring until it
> first encounters a node in DC1 or DC2 ? and then place the remaining
> replicas ?
The NTS considers each DC to have it's own ring. This can make token
selection in a multi DC environment confusing at times. There is something
in the DS docs about it.
Cheers
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 23/05/2012, at 3:16 PM, java jalwa wrote:
> Hi all,
> I am a bit confused regarding the terms "replica" and
> "replication factor". Assume that I am using RandomPartitioner and
> NetworkTopologyStrategy for replica placement.
> From what I understand, with a RandomPartitioner, a row key will
> always be hashed and be stored on the node that owns the range to
> which the key is mapped.
> http://www.datastax.com/docs/1.0/cluster_architecture/repli
Re: Does or will Cassandra support OpenJDK ?
I've used java-package under Debian (http://wiki.debian.org/JavaPackage) which turns your download from Oracle into a .deb. This may work on Ubuntu as well. On May 14, 2012, at 11:19 PM, aaron morton wrote: > To get the latest sun java 6 JRE on a ubuntu machine using apt-get I've used > the instructions here https://help.ubuntu.com/community/Java#JRE_only > > I've also use open JDK for java 6 on ubuntu without issue. You will want to > edit cassandra-env.sh to enable the jamm memory meter though, just comment > out the if statement and leave the JVM_OPS… line un commented. > > Cheers > > > - > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 15/05/2012, at 2:33 AM, Jeremiah Jordan wrote: > >> Open JDK is java 1.7. Once Cassandra supports Java 1.7 it would most likely >> work on Open JDK, as the 1.7 Open JDK really is the same thing as Oracle JDK >> 1.7 without some licensed stuff. >> >> -Jeremiah >> >> On May 11, 2012, at 10:02 PM, ramesh wrote: >> >>> I've had problem downloading the Sun (Oracle) JDK and found this thread >>> where the Oracle official is insisting or rather forcing Linux users to >>> move to OpenJDK. Here is the thread >>> >>> https://forums.oracle.com/forums/thread.jspa?threadID=2365607 >>> >>> I need this because I run Cassandra. >>> Just curious to know if I would be able to avoid the pain of using Sun JDK >>> in future for production Cassandra ? >>> >>> regards >>> Ramesh >> >
Re: Keyspace lost after restart
Conan,
Good to see I'm not alone in this! I just set up a fresh test cluster. I first
did a fresh install of 1.1.0 and was able to replicate the issue. I then did a
fresh install using 1.0.10 and didn't see the issue. So it looks like rolling
back to 1.0.10 could be the answer for now.
Jeff
On May 11, 2012, at 10:40 AM, Conan Cook wrote:
> Hi,
>
> OK we're pretty sure we dropped and re-created the keyspace before restarting
> the Cassandra nodes during some testing (we've been migrating to a new
> cluster). The keyspace was created via the cli:
>
>
> create keyspace m7
>
> with placement_strategy = 'NetworkTopologyStrategy'
>
> and strategy_options = {us-east: 3}
>
> and durable_writes = true;
>
> I'm pretty confident that it's a result of the issue I spotted before:
>
> https://issues.apache.org/jira/browse/CASSANDRA-4219
>
> Does anyone know whether this also affected versions before 1.1.0? If not
> then we can just roll back until there's a fix; we're not using our cluster
> in production so we can afford to just bin it all and load it again. +1 for
> this being a major issue though, the fact that you can't see it until you
> restart a node makes it quite dangerous, and that node is lost when it occurs
> (I also haven't been able to restore the schema in any way).
>
> Thanks very much,
>
>
> Conan
>
>
>
> On 10 May 2012 17:15, Conan Cook wrote:
> Hi Aaron,
>
> Thanks for getting back to me! Yes, I believe our keyspace was created prior
> to 1.1, and I think I also understand why you're asking that, having found
> this:
>
> https://issues.apache.org/jira/browse/CASSANDRA-4219
>
> Here's our startup log:
>
> https://gist.github.com/2654155
>
> There isn't much in there of interest however. It may well be the case that
> we created our keyspace, dropped it, then created it again. The dev
> responsible for setting it up is ill today, but I'll get back to you tomorrow
> with exact details of how it was originally created and whether we did
> definitely drop and re-create it.
>
> Ta,
>
> Conan
>
>
> On 10 May 2012 11:43, aaron morton wrote:
> Was this a schema that was created prior to 1.1 ?
>
> What process are you using to create the schema ?
>
> Can you share the logs from system startup ? Up until it logs "Listening for
> thrift clients". (if they are long please link to them)
>
> Cheers
>
> -
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 10/05/2012, at 1:04 AM, Conan Cook wrote:
>
>> Sorry, forgot to mention we're running Cassandra 1.1.
>>
>> Conan
>>
>> On 8 May 2012 17:51, Conan Cook wrote:
>> Hi Cassandra Folk,
>>
>> We've experienced a problem a couple of times where Cassandra nodes lose a
>> keyspace after a restart. We've restarted 2 out of 3 nodes, and they have
>> both experienced this problem; clearly we're doing something wrong, but
>> don't know what. The data files are all still there, as before, but the
>> node can't see the keyspace (we only have one). Tthe nodetool still says
>> that each one is responsible for 33% of the keys, but the disk usage has
>> dropped to a tiny amount on the nodes that we've restarted. I saw this:
>>
>> http://mail-archives.apache.org/mod_mbox/cassandra-user/201202.mbox/%3c4f3582e7.20...@conga.com%3E
>>
>> Seems to be exactly our problem, but we have not modified the cassandra.yaml
>> - we have overwritten it through an automated process, and that happened
>> just before restarting, but the contents did not change.
>>
>> Any ideas as to what might cause this, or how the keyspace can be restored
>> (like I say, the data is all still in the data directory).
>>
>> We're running in AWS.
>>
>> Thanks,
>>
>>
>> Conan
>>
>
>
>
Re: Problem with Keyspace after drop
I have tried restarting all of the nodes with no change. Created JIRA https://issues.apache.org/jira/browse/CASSANDRA-4219 Thanks, Jeff On May 3, 2012, at 9:46 PM, Sylvain Lebresne wrote: > Would you mind opening a ticket on JIRA > (https://issues.apache.org/jira/browse/CASSANDRA). This is > definitively a bug and it's be easier to follow up there. > > Did you try restarting the node on which you try the create keyspace? > > -- > Sylvain > > On Thu, May 3, 2012 at 8:43 PM, Jeff Williams > wrote: >> As a follow up, I am running version 1.1 and have tried nodetool repair, >> cleanup, compact. I can create other keyspaces, but still can't create a >> keyspace called PlayLog even though it is not listed anywhere. >> >> On May 3, 2012, at 6:35 PM, Jeff Williams wrote: >> >>> Hi, >>> >>> I'm doing testing and wanted to drop a keyspace (with a column family) to >>> re-add it with a different strategy. So I ran in cqlsh: >>> >>> DROP KEYSPACE PlayLog; >>> >>> CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' >>> AND strategy_options:replication_factor = 2; >>> >>> And everything seemed to be fine. I ran some inserts, which also seemed to >>> go fine, but then selecting them gave me: >>> >>> cqlsh:PlayLog> select count(*) from playlog; >>> TSocket read 0 bytes >>> >>> I wasn't sure what was wrong, so I tried dropping and creating again, and >>> now when I try to create I get: >>> >>> cqlsh> CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' >>> ... AND strategy_options:replication_factor = 2; >>> TSocket read 0 bytes >>> >>> And the keyspace doesn't get created. In the log it shows: >>> >>> ERROR [Thrift:4] 2012-05-03 18:23:05,124 CustomTThreadPoolServer.java (line >>> 204) Error occurred during processing of message. >>> java.lang.RuntimeException: java.util.concurrent.ExecutionException: >>> java.lang.AssertionError >>>at >>> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372) >>>at >>> org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191) >>>at >>> org.apache.cassandra.service.MigrationManager.announceNewKeyspace(MigrationManager.java:129) >>>at >>> org.apache.cassandra.cql.QueryProcessor.processStatement(QueryProcessor.java:701) >>>at >>> org.apache.cassandra.cql.QueryProcessor.process(QueryProcessor.java:875) >>>at >>> org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1235) >>>at >>> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3458) >>>at >>> org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3446) >>>at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) >>>at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) >>>at >>> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186) >>>at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown >>> Source) >>>at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >>>at java.lang.Thread.run(Unknown Source) >>> Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError >>>at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) >>>at java.util.concurrent.FutureTask.get(Unknown Source) >>>at >>> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) >>>... 13 more >>> Caused by: java.lang.AssertionError >>>at >>> org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441) >>>at >>> org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339) >>>at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269) >>>at >>> org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214) >>>at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) >>>at java.util.concurrent.FutureTask.run(Unknown Source) >>>... 3 more >>> ERROR [MigrationStage:1] 2012-05-03 18:23:05,124 >>> AbstractCassandraDaemon.
Re: Problem with Keyspace after drop
As a follow up, I am running version 1.1 and have tried nodetool repair, cleanup, compact. I can create other keyspaces, but still can't create a keyspace called PlayLog even though it is not listed anywhere. On May 3, 2012, at 6:35 PM, Jeff Williams wrote: > Hi, > > I'm doing testing and wanted to drop a keyspace (with a column family) to > re-add it with a different strategy. So I ran in cqlsh: > > DROP KEYSPACE PlayLog; > > CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' > AND strategy_options:replication_factor = 2; > > And everything seemed to be fine. I ran some inserts, which also seemed to go > fine, but then selecting them gave me: > > cqlsh:PlayLog> select count(*) from playlog; > TSocket read 0 bytes > > I wasn't sure what was wrong, so I tried dropping and creating again, and now > when I try to create I get: > > cqlsh> CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' > ... AND strategy_options:replication_factor = 2; > TSocket read 0 bytes > > And the keyspace doesn't get created. In the log it shows: > > ERROR [Thrift:4] 2012-05-03 18:23:05,124 CustomTThreadPoolServer.java (line > 204) Error occurred during processing of message. > java.lang.RuntimeException: java.util.concurrent.ExecutionException: > java.lang.AssertionError >at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372) >at > org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191) >at > org.apache.cassandra.service.MigrationManager.announceNewKeyspace(MigrationManager.java:129) >at > org.apache.cassandra.cql.QueryProcessor.processStatement(QueryProcessor.java:701) >at > org.apache.cassandra.cql.QueryProcessor.process(QueryProcessor.java:875) >at > org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1235) >at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3458) >at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3446) >at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) >at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) >at > org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186) >at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown > Source) >at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >at java.lang.Thread.run(Unknown Source) > Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError >at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) >at java.util.concurrent.FutureTask.get(Unknown Source) >at > org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) >... 13 more > Caused by: java.lang.AssertionError >at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441) >at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339) >at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269) >at > org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214) >at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) >at java.util.concurrent.FutureTask.run(Unknown Source) >... 3 more > ERROR [MigrationStage:1] 2012-05-03 18:23:05,124 AbstractCassandraDaemon.java > (line 134) Exception in thread Thread[MigrationStage:1,5,main] > java.lang.AssertionError >at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441) >at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339) >at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269) >at > org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214) >at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) >at java.util.concurrent.FutureTask.run(Unknown Source) >at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown > Source) >at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >at java.lang.Thread.run(Unknown Source) > > Any ideas how I can recover from this? > > Jeff >
Re: Write performance compared to Postgresql
Just to follow this up, I repeated the test with a multi-threaded java (Hector) client and was able to get much better performance - 10,000 rows in just over a second. So it looks like the client latency was the killer and I have since read that the ruby thrift implementation is not the fastest. On Apr 4, 2012, at 9:11 AM, Jeff Williams wrote: > On three machines on the same subnet as the two cassandra nodes. > > On Apr 3, 2012, at 6:40 PM, Collard, David L (Dave) wrote: > >> Where is your client running? >> >> -Original Message- >> From: Jeff Williams [mailto:je...@wherethebitsroam.com] >> Sent: Tuesday, April 03, 2012 11:09 AM >> To: user@cassandra.apache.org >> Subject: Re: Write performance compared to Postgresql >> >> Vitalii, >> >> Yep, that sounds like a good idea. Do you have any more information about >> how you're doing that? Which client? >> >> Because even with 3 concurrent client nodes, my single postgresql server is >> still out performing my 2 node cassandra cluster, although the gap is >> narrowing. >> >> Jeff >> >> On Apr 3, 2012, at 4:08 PM, Vitalii Tymchyshyn wrote: >> >>> Note that having tons of TCP connections is not good. We are using async >>> client to issue multiple calls over single connection at same time. You can >>> do the same. >>> >>> Best regards, Vitalii Tymchyshyn. >>> >>> 03.04.12 16:18, Jeff Williams написав(ла): >>>> Ok, so you think the write speed is limited by the client and protocol, >>>> rather than the cassandra backend? This sounds reasonable, and fits with >>>> our use case, as we will have several servers writing. However, a bit >>>> harder to test! >>>> >>>> Jeff >>>> >>>> On Apr 3, 2012, at 1:27 PM, Jake Luciani wrote: >>>> >>>>> Hi Jeff, >>>>> >>>>> Writing serially over one connection will be slower. If you run many >>>>> threads hitting the server at once you will see throughput improve. >>>>> >>>>> Jake >>>>> >>>>> >>>>> >>>>> On Apr 3, 2012, at 7:08 AM, Jeff Williams >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am looking at cassandra for a logging application. We currently log to >>>>>> a Postgresql database. >>>>>> >>>>>> I set up 2 cassandra servers for testing. I did a benchmark where I had >>>>>> 100 hashes representing logs entries, read from a json file. I then >>>>>> looped over these to do 10,000 log inserts. I repeated the same writing >>>>>> to a postgresql instance on one of the cassandra servers. The script is >>>>>> attached. The cassandra writes appear to perform a lot worse. Is this >>>>>> expected? >>>>>> >>>>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>>>> cassandra >>>>>> 3.17 0.48 3.65 ( 12.032212) >>>>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>>>> postgres >>>>>> 2.14 0.33 2.47 ( 7.002601) >>>>>> >>>>>> Regards, >>>>>> Jeff >>>>>> >>>>>> >>> >> >
Problem with Keyspace after drop
Hi, I'm doing testing and wanted to drop a keyspace (with a column family) to re-add it with a different strategy. So I ran in cqlsh: DROP KEYSPACE PlayLog; CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' AND strategy_options:replication_factor = 2; And everything seemed to be fine. I ran some inserts, which also seemed to go fine, but then selecting them gave me: cqlsh:PlayLog> select count(*) from playlog; TSocket read 0 bytes I wasn't sure what was wrong, so I tried dropping and creating again, and now when I try to create I get: cqlsh> CREATE KEYSPACE PlayLog WITH strategy_class = 'SimpleStrategy' ... AND strategy_options:replication_factor = 2; TSocket read 0 bytes And the keyspace doesn't get created. In the log it shows: ERROR [Thrift:4] 2012-05-03 18:23:05,124 CustomTThreadPoolServer.java (line 204) Error occurred during processing of message. java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.AssertionError at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372) at org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191) at org.apache.cassandra.service.MigrationManager.announceNewKeyspace(MigrationManager.java:129) at org.apache.cassandra.cql.QueryProcessor.processStatement(QueryProcessor.java:701) at org.apache.cassandra.cql.QueryProcessor.process(QueryProcessor.java:875) at org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1235) at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3458) at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3446) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: java.lang.AssertionError at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source) at java.util.concurrent.FutureTask.get(Unknown Source) at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368) ... 13 more Caused by: java.lang.AssertionError at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441) at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339) at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269) at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) ... 3 more ERROR [MigrationStage:1] 2012-05-03 18:23:05,124 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[MigrationStage:1,5,main] java.lang.AssertionError at org.apache.cassandra.db.DefsTable.updateKeyspace(DefsTable.java:441) at org.apache.cassandra.db.DefsTable.mergeKeyspaces(DefsTable.java:339) at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:269) at org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Any ideas how I can recover from this? Jeff
Re: Write performance compared to Postgresql
On three machines on the same subnet as the two cassandra nodes. On Apr 3, 2012, at 6:40 PM, Collard, David L (Dave) wrote: > Where is your client running? > > -Original Message- > From: Jeff Williams [mailto:je...@wherethebitsroam.com] > Sent: Tuesday, April 03, 2012 11:09 AM > To: user@cassandra.apache.org > Subject: Re: Write performance compared to Postgresql > > Vitalii, > > Yep, that sounds like a good idea. Do you have any more information about how > you're doing that? Which client? > > Because even with 3 concurrent client nodes, my single postgresql server is > still out performing my 2 node cassandra cluster, although the gap is > narrowing. > > Jeff > > On Apr 3, 2012, at 4:08 PM, Vitalii Tymchyshyn wrote: > >> Note that having tons of TCP connections is not good. We are using async >> client to issue multiple calls over single connection at same time. You can >> do the same. >> >> Best regards, Vitalii Tymchyshyn. >> >> 03.04.12 16:18, Jeff Williams написав(ла): >>> Ok, so you think the write speed is limited by the client and protocol, >>> rather than the cassandra backend? This sounds reasonable, and fits with >>> our use case, as we will have several servers writing. However, a bit >>> harder to test! >>> >>> Jeff >>> >>> On Apr 3, 2012, at 1:27 PM, Jake Luciani wrote: >>> >>>> Hi Jeff, >>>> >>>> Writing serially over one connection will be slower. If you run many >>>> threads hitting the server at once you will see throughput improve. >>>> >>>> Jake >>>> >>>> >>>> >>>> On Apr 3, 2012, at 7:08 AM, Jeff Williams >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I am looking at cassandra for a logging application. We currently log to >>>>> a Postgresql database. >>>>> >>>>> I set up 2 cassandra servers for testing. I did a benchmark where I had >>>>> 100 hashes representing logs entries, read from a json file. I then >>>>> looped over these to do 10,000 log inserts. I repeated the same writing >>>>> to a postgresql instance on one of the cassandra servers. The script is >>>>> attached. The cassandra writes appear to perform a lot worse. Is this >>>>> expected? >>>>> >>>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>>> cassandra >>>>> 3.17 0.48 3.65 ( 12.032212) >>>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>>> postgres >>>>> 2.14 0.33 2.47 ( 7.002601) >>>>> >>>>> Regards, >>>>> Jeff >>>>> >>>>> >> >
Re: Write performance compared to Postgresql
Vitalii, Yep, that sounds like a good idea. Do you have any more information about how you're doing that? Which client? Because even with 3 concurrent client nodes, my single postgresql server is still out performing my 2 node cassandra cluster, although the gap is narrowing. Jeff On Apr 3, 2012, at 4:08 PM, Vitalii Tymchyshyn wrote: > Note that having tons of TCP connections is not good. We are using async > client to issue multiple calls over single connection at same time. You can > do the same. > > Best regards, Vitalii Tymchyshyn. > > 03.04.12 16:18, Jeff Williams написав(ла): >> Ok, so you think the write speed is limited by the client and protocol, >> rather than the cassandra backend? This sounds reasonable, and fits with our >> use case, as we will have several servers writing. However, a bit harder to >> test! >> >> Jeff >> >> On Apr 3, 2012, at 1:27 PM, Jake Luciani wrote: >> >>> Hi Jeff, >>> >>> Writing serially over one connection will be slower. If you run many >>> threads hitting the server at once you will see throughput improve. >>> >>> Jake >>> >>> >>> >>> On Apr 3, 2012, at 7:08 AM, Jeff Williams >>> wrote: >>> >>>> Hi, >>>> >>>> I am looking at cassandra for a logging application. We currently log to a >>>> Postgresql database. >>>> >>>> I set up 2 cassandra servers for testing. I did a benchmark where I had >>>> 100 hashes representing logs entries, read from a json file. I then looped >>>> over these to do 10,000 log inserts. I repeated the same writing to a >>>> postgresql instance on one of the cassandra servers. The script is >>>> attached. The cassandra writes appear to perform a lot worse. Is this >>>> expected? >>>> >>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>> cassandra >>>> 3.17 0.48 3.65 ( 12.032212) >>>> jeff@transcoder01:~$ ruby cassandra-bm.rb >>>> postgres >>>> 2.14 0.33 2.47 ( 7.002601) >>>> >>>> Regards, >>>> Jeff >>>> >>>> >
Re: Write performance compared to Postgresql
Ok, so you think the write speed is limited by the client and protocol, rather than the cassandra backend? This sounds reasonable, and fits with our use case, as we will have several servers writing. However, a bit harder to test! Jeff On Apr 3, 2012, at 1:27 PM, Jake Luciani wrote: > Hi Jeff, > > Writing serially over one connection will be slower. If you run many threads > hitting the server at once you will see throughput improve. > > Jake > > > > On Apr 3, 2012, at 7:08 AM, Jeff Williams wrote: > >> Hi, >> >> I am looking at cassandra for a logging application. We currently log to a >> Postgresql database. >> >> I set up 2 cassandra servers for testing. I did a benchmark where I had 100 >> hashes representing logs entries, read from a json file. I then looped over >> these to do 10,000 log inserts. I repeated the same writing to a postgresql >> instance on one of the cassandra servers. The script is attached. The >> cassandra writes appear to perform a lot worse. Is this expected? >> >> jeff@transcoder01:~$ ruby cassandra-bm.rb >> cassandra >> 3.17 0.48 3.65 ( 12.032212) >> jeff@transcoder01:~$ ruby cassandra-bm.rb >> postgres >> 2.14 0.33 2.47 ( 7.002601) >> >> Regards, >> Jeff >> >>
Write performance compared to Postgresql
Hi,
I am looking at cassandra for a logging application. We currently log to a
Postgresql database.
I set up 2 cassandra servers for testing. I did a benchmark where I had 100
hashes representing logs entries, read from a json file. I then looped over
these to do 10,000 log inserts. I repeated the same writing to a postgresql
instance on one of the cassandra servers. The script is attached. The cassandra
writes appear to perform a lot worse. Is this expected?
jeff@transcoder01:~$ ruby cassandra-bm.rb
cassandra
3.17 0.48 3.65 ( 12.032212)
jeff@transcoder01:~$ ruby cassandra-bm.rb
postgres
2.14 0.33 2.47 ( 7.002601)
Regards,
Jeff
require 'rubygems'
require 'cassandra-cql'
require 'simple_uuid'
require 'benchmark'
require 'json'
require 'active_record'
type = 'postgres'
#type = 'cassandra'
puts type
ActiveRecord::Base.establish_connection(
#:adapter => "jdbcpostgresql",
:adapter => "postgresql",
:host => "meta01",
:username => "postgres",
:database => "test")
db = nil
if type == 'postgres'
db = ActiveRecord::Base.connection
else
db = CassandraCQL::Database.new('meta01:9160', {:keyspace => 'PlayLog'})
end
def cql_insert(table, key, key_value)
cql = "INSERT INTO #{table} (KEY, "
cql << key_value.keys.join(', ')
cql << ") VALUES ('#{key}', "
cql << (key_value.values.map {|x| "'#{x}'" }).join(', ')
cql << ")"
cql
end
def quote_value(x, type=nil)
if x.nil?
return 'NULL'
else
return "'#{x}'"
end
end
def sql_insert(table, key_value)
key_value.delete('time')
cql = "INSERT INTO #{table} ("
cql << key_value.keys.join(', ')
cql << ") VALUES ("
cql << (key_value.values.map {|x| quote_value(x) }).join(', ')
cql << ")"
cql
end
# load 100 hashes of log details
rows = []
File.open('data.json') do |f|
rows = JSON.load(f)
end
bm = Benchmark.measure do
(1..1).each do |i|
row = rows[i%100]
if type == 'postgres'
fred = sql_insert('playlog', row)
else
fred = cql_insert('playlog', SimpleUUID::UUID.new.to_guid, row)
end
db.execute(fred)
end
end
puts bm
