RE: Backups in Cassandra

2019-08-07 Thread Rhys.Campbell
Just to add to this… We do snapshot, incremental and commitlog backups along with schema and config backups. All is copied to S3 although we do keep a small number of snapshots / inc / commitlog on the local node in the rare event they are needed. We have written some Ansible to restore the who

Repairs/compactions on tables with solr indexes

2019-08-07 Thread Ayub M
Hello, we are using DSE Search workload with Search and Cass running on same nodes/jvm. 1. When repairs are run, does it initiate rebuilds of solr indexes? Does it rebuild only when any data is repaired? 2. How about the compactions, does it trigger any search indexes rebuilds? I guess not, since

Re: Backups in Cassandra

2019-08-07 Thread cclive1601你
We have also made backup and restore for Apache Cassandra,backup process are 1.do incremental backup for flushed sstable ;do incremental backup for commitlog ; 2.do snapshot for the cluster periodically,also meta info are needed to backup(token and table info); 3.for exception like node joining and

Re: Backups in Cassandra

2019-08-07 Thread Connor Lin
Hi Krish, It is recommended to have backups. Although I haven't practiced it myself, but I find this might be helpful. https://thelastpickle.com/blog/2018/04/03/cassandra-backup-and-restore-aws-ebs.html Sincerely yours, Connor Lin On Thu, Aug 8, 2019 at 5:47 AM Krish Donald wrote: > Hi Folks

Backups in Cassandra

2019-08-07 Thread Krish Donald
Hi Folks, First question is , Do you take backup for your cassandra cluster ? If answer is yes then question follows: 1. How do you take backup ? 1.1 ) Is it only snapshot? 1.2 ) We are on AWS with very large cluster around 51 nodes with 1TB data on each node.

Re: Datafile Corruption

2019-08-07 Thread Nitan Kainth
Repair during upgrade have caused corruption too. Also, dropping and adding columns with same name but different type Regards, Nitan Cell: 510 449 9629 > On Aug 7, 2019, at 2:42 PM, Jeff Jirsa wrote: > > Is compression enabled? > > If not, bit flips on disk can corrupt data files and reads +

Re: Datafile Corruption

2019-08-07 Thread Jeff Jirsa
Is compression enabled? If not, bit flips on disk can corrupt data files and reads + repair may send that corruption to other hosts in the cluster > On Aug 7, 2019, at 3:46 AM, Philip Ó Condúin wrote: > > Hi All, > > I am currently experiencing multiple datafile corruptions across most node

Re: Datafile Corruption

2019-08-07 Thread Laxmikant Upadhyay
Few for reasons: Sudden Power cut Disk full Issue in casandra version like Cassandra-13752 On Wed, Aug 7, 2019, 4:16 PM Philip Ó Condúin wrote: > Hi All, > > I am currently experiencing multiple datafile corruptions across most > nodes in my cluster, there seems to be no pattern to the corruptio

Point in time restore not working when primary key is blob

2019-08-07 Thread Shaurya Gupta
Hi, I'm trying to do a point in time restore using commit logs. It seems to be working fine if I have primary as text but it does not work and just restores all the rows if the primary key is blob (which is default if I create a keyspace using cassandra-stress). Is this a known issue? Thanks --

Datafile Corruption

2019-08-07 Thread Philip Ó Condúin
Hi All, I am currently experiencing multiple datafile corruptions across most nodes in my cluster, there seems to be no pattern to the corruption. I'm starting to think it might be a bug, we're using Cassandra 2.2.13. Without going into detail about the issue I just want to confirm something. C