Re:

Kurt, thank you very much for your answer! Your remark on GC totally changed my thoughts on cassandra resources usage. So.. more questions to the respective audience underway. What is generally considered as 1) "too large" page size, 2)"large" page size 3) "normal conditions" page size? How exa

[no subject]

Hello there, Let me get straight to the point. Yesterday our three node Cassandra production cluster had a problem and we could not find a solution yet. Before taking more radical actions, I would like to consult you about the issue. We are using Cassandra version 3.11.0. Cluster is living on AWS

Tombstone

Hi all, We using Cassandra for storing events which are time series based for batch processing once a particular batch based on hour is processed we delete the entries but we were left with almost 18% deletes marked as Tombstones. I ran compaction on the particular CF to

Re: saving distinct data in cassandra result in many tombstones

1. How to use sharding partition key in a way that partitions end up in different nodes? You could, for example, create a table with a bucket column added to the partition key: Table distinct( hourNumber int, bucket int, //could be a 5 minute bucket for example key text, distinctValue long primary

Cassandra Client Program not Working with NettySSLOptions

Hi, I referred to this link https://docs.datastax.com/en/developer/java-driver/3.0/manual/ssl/ to implement a simple Cassandra client using datastax driver 3.0.0 on SSL with OpenSSL options but unable to run it. Getting generic

Re: Tombstone

TimeWindowCompactionStrategy and don’t delete the data you should be relying on Cassandra to drop the SSTables once the data inside has expired. THat 18% is probably waiting on gc_grace, this shouldn’t be an issue if you are letting TWCS drop the data rather then running deletes. Regards, Evely

Re:

Check with Java Using version. On Tue, Jun 19, 2018 at 6:18 PM, Deniz Acay wrote: > Hello there, > > Let me get straight to the point. Yesterday our three node Cassandra > production cluster had a problem and we could not find a solution yet. > Before taking more radical actions, I would like to

Re: RE: [EXTERNAL] Cluster is unbalanced

>If it was partition key issue, we would see similar number of partition keys across nodes. If we look closely number of keys across nodes vary a lot. I'm not sure about that, is it possible you're writing more new partitions to some nodes even though each node owns the same number of tokens? On

Re:

Deniz, The assertion error you're seeing appears to be coming from this line: https://github.com/apache/cassandra/blob/cassandra-3.11.0/src/java/org/apache/cassandra/db/lifecycle/LogReplicaSet.java#L63 This file describes a LogReplicaSet as "A set of physical files on disk, [where] each file is a

Re: Cassandra Client Program not Working with NettySSLOptions

Is the server configured to use encryption? On Tue, Jun 19, 2018 at 3:59 AM Jahar Tyagi wrote: > Hi, > > I referred to this link > https://docs.datastax.com/en/developer/java-driver/3.0/manual/ssl/ > to > implement a simple Ca

Re: RE: [EXTERNAL] Cluster is unbalanced

We do not chose the node where partition will go. I thought it is snitch's role to chose replica nodes. Even the partition size does not vary on our largest column family: Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count                            

Re: Timestamp on hints file and system.hints table data

Kurt, There are no errors in system.log. Just messages about hints sent and played. We will try stop and delete solution and let you know if that fixed the problem. But my curiosity is more about timestamp on hints files, why do we have files with back date; even though I deleted files manually

Re: Tombstone

The most likely explanation is tombstones in files that won’t be collected as they potentially overlap data in other files with a lower timestamp (especially true if your partition key doesn’t change and you’re writing and deleting data within a partition) -- Jeff Jirsa > On Jun 19, 2018, at

Re:

Just assume that the rows you read in a page all end up in the heap at the same time If you’re reading 1000 rows of 100 bytes, no big deal, you’ve got 100kb per read thread on the heap If you’re reading 100 1mb rows, now you’ve got 100MB per thread on the heap Assuming an 8gb heap with 2gb you

Re: Tombstone

The Partition key is made of datetime(basically date truncated to hour) and bucket.I think your RCA may be correct since we are deleting the partition rows one by one not in a batch files maybe overlapping for the particular partition.A scheduled thread picks the rows for a p

Re:

I checked file system integrity and contents. Everything seems OK. Whatever the reason, that problem spread to all nodes at the same time, so I believe it is not very likely for all volumes or any hardware to fail at the same time. On Tue, Jun 19, 2018 at 2:25 PM Joshua Galbraith wrote: > Deniz

Re:

It's in the middle of trying to replay the unfinished compaction logs - there's a file in your data directories that will end with .log - can you paste the contents somewhere like pastebin/gist? On Tue, Jun 19, 2018 at 7:42 AM, Deniz Acay wrote: > I checked file system integrity and contents.

Re: 3.11.2 memory leak

Hi All, Do we kmow whats the stable version for now if u wish to upgrade ? On Tuesday, June 5, 2018, Steinmaurer, Thomas < thomas.steinmau...@dynatrace.com> wrote: > Jeff, > > > > FWIW, when talking about https://issues.apache.org/ > jira/browse/CASSANDRA-13929, there is a patch available since

RE: [EXTERNAL] Re: Tombstone

This sounds like a queue pattern, which is typically an anti-pattern for Cassandra. I would say that it is very difficult to get the access patterns, tombstones, and everything else lined up properly to solve a queue problem. Sean Durity From: Abhishek Singh Sent: Tuesday, June 19, 2018 10:41

RE: RE: [EXTERNAL] Cluster is unbalanced

You are correct that the cluster decides where data goes (based on the hash of the partition key). However, if you choose a “bad” partition key, you may not get good distribution of the data, because the hash is deterministic (it always goes to the same nodes/replicas). For example, if you have

Re: How do you monitoring Cassandra Cluster?

Hi Felipe, You can use this project https://github.com/criteo/cassandra_exporter if you are using Prometheus (Disclamer, I am one of the author of it). There is included a Grafana dashboard that aggregate metrics per cluster for you, and in the "edit view" of each chart there are hidden queries

Re: RE: RE: [EXTERNAL] Cluster is unbalanced

Hi Sean, Here is create table: CREATE TABLE ks.cf (     id text PRIMARY KEY,     accessdata blob ) WITH bloom_filter_fp_chance = 0.01     AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}     AND comment = ''     AND compaction = {'class': 'org.apache.cassandra.db.compaction.Size

Re:

Yes there is a file like that. It contains a single line: ADD:[/var/data/cassandra/data/[company_name]/[table_name]-3998ed90e01811e7820af142a4a9c0fd/mc-28797-big,0,8][3557304334] On Tue, Jun 19, 2018 at 6:07 PM Jeff Jirsa wrote: > It's in the middle of trying to replay the unfinished compaction

sstableloader from dse 4.8.4 to apache cassandra 3.11.1

Hello, I’m trying to use sstablloader from dse 4.8.4( 2.1.12) to apache 3.11.1, i’m getting below error. but works fine when i use stableloader dse 5.1.2(apache 3.11.0) Could not retrieve endpoint ranges: java.io.IOException: Failed to open transport to: host-ip:9160. Any work around to use

Re: sstableloader from dse 4.8.4 to apache cassandra 3.11.1

Never mind found it. its not a supported version. > On Jun 19, 2018, at 2:41 PM, rajpal reddy wrote: > > > Hello, > > I’m trying to use sstablloader from dse 4.8.4( 2.1.12) to apache 3.11.1, i’m > getting below error. but works fine when i use stableloader dse 5.1.2(apache > 3.11.0) > Could

how to avoid lightwieght transactions

Hi all, we have a use case where we need to update frequently our rows. Now in order to do so and so that we dont override updates we have to resort to lightweight transactions. Since lightweight is expensive(could be 4 times as expensive as normal insert) , how do we model around it. e.g i have a

Re: RE: RE: [EXTERNAL] Cluster is unbalanced

> id text PRIMARY KEY What values are written to this id field? Can you give us some examples or explain the general use case? On Tue, Jun 19, 2018 at 1:18 PM, learner dba wrote: > Hi Sean, > > Here is create table: > > CREATE TABLE ks.cf ( > > id text PRIMARY KEY, > > accessdata blob >

Re: 3.11.2 memory leak

At this point I'd wait for 3.11.3. If you can't, you can get away with backporting a few repair fixes or just doing sub range repairs on 3.11.2 On Wed., 20 Jun. 2018, 01:10 Abdul Patel, wrote: > Hi All, > > Do we kmow whats the stable version for now if u wish to upgrade ? > > On Tuesday, June 5

Re: Cluster is unbalanced

what is your keyspace configuration. Do you have all the keyspaces configured for both DCs? can you run below query from cqlsh and see if the keyspace is configured to use both DCs select * from system.schema_keyspaces; # if your cluster is on 2.1 or less select * from system_schema.keyspaces