Re: Verify cassandra backup and restore in C * 2.1

2016-08-08 Thread INDRANIL BASU
Thanks Rajath. We have atleast 20 millions of records, so count(*) with limit and token is getting cumbersome.  Yes using nodetool snapshot. I know there is a nodetool verify, but i m using C * 2.1 Copy commands sometimes fails as well. So any better easy mechanism?  Thanks and regards,-- Indran

Re: a solution of getting cassandra cross-datacenter latency at a certain time

2016-08-08 Thread Stone Fang
thanks for you response. @Ryan Svihla,yeah you are right,there will be a ttl.only store most recent data. @Chris good idea on getting the instantaneous cross-datacenter latency.but seems still have issues. 1.based on application insert/update record,not a regular generate record.once the networ

Re: Adding Materialized View triggers "Mutation Too Large" error.

2016-08-08 Thread Jason J. W. Williams
Hi Paulo, Thanks for bug link! I've subscribed to it for updates. Thankfully, these tables for something still in beta so we can wipe the data. But it'll be great when its fixed. -J On Mon, Aug 8, 2016 at 4:26 PM, Paulo Motta wrote: > What happens is that when trying to rebuild the MV, the reb

Re: Adding Materialized View triggers "Mutation Too Large" error.

2016-08-08 Thread Paulo Motta
What happens is that when trying to rebuild the MV, the rebuilder tries to create a very large batch that exceeds commitlog_segment_size_in_mb. This limitation is currently being addressed on CASSANDRA-11670. Two options I can see to workaround this for now: 1) increase commitlog_segment_size_in_mb

Adding Materialized View triggers "Mutation Too Large" error.

2016-08-08 Thread Jason J. W. Williams
HI Guys, We're running Cassandra 3.0.8, and needed to add a field to a table and it's materialized views. We dropped the MVs, added the fields to the underlying table, and recreated MVs with the new fields. However, the MV creation is failing with: WARN [CompactionExecutor:6933] 2016-08-08 22:4

Re: compactions stuck and restart doesn't kill it

2016-08-08 Thread Surbhi Gupta
I have faced similar kind of issue while we were using LCS ... We ran the scrub and it was helpful to clear the corruption and compaction caught up well after that ... On 8 August 2016 at 13:23, John Wong wrote: > Hi. > > I don't see any error logs indicate corrupted sstable error. Is it safe t

Re: compactions stuck and restart doesn't kill it

2016-08-08 Thread John Wong
Hi. I don't see any error logs indicate corrupted sstable error. Is it safe to run this? Thanks. On Mon, Aug 8, 2016 at 1:18 PM, Surbhi Gupta wrote: > Can you see if any of the sstable is corrupt ? > I have seen with my past experience , if any of the sstable which is part > of the compaction

Re: Doing rolling upgrades on two DCs

2016-08-08 Thread Erik Forkalsud
The low-risk approach would be to change one thing at a time, but I don't think there's a technical reason you couldn't change the commit log and data directory locations while updating to a new version. - EF On 08/08/2016 07:58 AM, Timon Wong wrote: Thank you! We would do a patch level upg

Re: Verify cassandra backup and restore in C * 2.1

2016-08-08 Thread Rajath Subramanyam
Hi Indranil, One approach is to do a row count on the original source table and the table that is restored from backup. How are you backing up data ? I am assuming you are issuing snapshot commands (either incremental or otherwise). I hope this helps. - Rajath Rajath Su

Re: compactions stuck and restart doesn't kill it

2016-08-08 Thread Surbhi Gupta
Can you see if any of the sstable is corrupt ? I have seen with my past experience , if any of the sstable which is part of the compaction is corrupt then compaction kind of hangs. Try to run nodetool scrub ... On 8 August 2016 at 09:34, John Wong wrote: > On Mon, Aug 8, 2016 at 11:16 AM, Surbh

Re: setting ttl using timestamp field of record during insertion in cassandra

2016-08-08 Thread Sujeet Kumar
Thanks.Actually i want to start ttl from the record timestamp field, not from insertion time(ttl starts from insertion time). On Mon, Aug 8, 2016 at 9:36 PM, wrote: > Perhaps you can set the default TTL when you create the table instead of > on every insert: > > http://docs.datastax.com/en/cql/3

Re: compactions stuck and restart doesn't kill it

2016-08-08 Thread John Wong
On Mon, Aug 8, 2016 at 11:16 AM, Surbhi Gupta wrote: > Once you restart a node compaction will start automatically, if u dont > want to do so then do > nodetool disableautocompaction as soon as node is started . > > Thanks. I certainly can give that a try for the specific column family, but even

RE: setting ttl using timestamp field of record during insertion in cassandra

2016-08-08 Thread SEAN_R_DURITY
Perhaps you can set the default TTL when you create the table instead of on every insert: http://docs.datastax.com/en/cql/3.1/cql/cql_reference/tabProp.html (see default_time_to_live property) Then no need for UDF. Sean Durity lord of the (C*) rings (Lead Cassandra Admin) Big DATA Team MTC 2250

Re: compactions stuck and restart doesn't kill it

2016-08-08 Thread Surbhi Gupta
Once you restart a node compaction will start automatically, if u dont want to do so then do nodetool disableautocompaction as soon as node is started . On 8 August 2016 at 07:22, John Wong wrote: > Hi > > We have a compaction stuck. No progress ever made. > > nodetool compactionstats > pending

setting ttl using timestamp field of record during insertion in cassandra

2016-08-08 Thread Sujeet Kumar
Hi I want to set TTL 30 days from the time field of table record(id,name,time) during insertion.For this I am creating a User Defined Function(UDF) bigint fun(rTime,cTime) as CREATE FUNCTION fun(rtime timestamp,ctime timestamp) CALLED ON NULL INPUT RETURNS bigint LANGUAGE java as 'return 25920

Re: Doing rolling upgrades on two DCs

2016-08-08 Thread Timon Wong
Thank you! We would do a patch level upgrade, from 2.2.1 to 2.2.7 (the latest stable). Besides, we want to adjust both `commitlog_directory` and `data_file_directories` settings (separate them into two better IOPS volumes), and we assumed they should be done before or after upgrade? On Sun, Aug 7,

compactions stuck and restart doesn't kill it

2016-08-08 Thread John Wong
Hi We have a compaction stuck. No progress ever made. nodetool compactionstats pending tasks: 2 compaction typekeyspace column family completed total unit progress Compactionmy_columnfamily 0 410383527 bytes 0.00% Active

Re: a solution of getting cassandra cross-datacenter latency at a certain time

2016-08-08 Thread Chris Lohfink
If you invoke the values operation on the mbean every minute (or whatever period) you can get a histogram of the cross dc the latencies. Just keep track of the values of each bin in histogram and look at the delta from previous time to the current time to find how many latencies occurred in each bi

Re: a solution of getting cassandra cross-datacenter latency at a certain time

2016-08-08 Thread Ryan Svihla
The first issue I can think of is the Latency table, if I understand you correctly, has an unbounded size for the partition key of DC and will over time just get larger as more measurements are recorded. Regards, Ryan Svihla > On Aug 8, 2016, at 2:58 AM, Stone Fang wrote: > > objective:get c

Verify cassandra backup and restore in C * 2.1

2016-08-08 Thread INDRANIL BASU
Hello,    How can we verify that cassandra data is backed up and restored correctly? -- IB

Re: Stale value appears after consecutive TRUNCATE

2016-08-08 Thread horschi
Hi Yuji, can you reproduce the behaviour with a single node? The reason I ask is because I probably have the same issue with my automated tests (which run truncate between every test), which run on my local laptop. Maybe around 5 tests randomly fail out of my 1800. I can see that the failed test

Spark/Cassandra/Zeppelin integration: How to

2016-08-08 Thread DuyHai Doan
Hello all For those who are struggling to integrate those 3 stacks (and I know there are quite a few, given the number of questions I got on this topic), I have written a blog post to explain the steps to make it work: http://www.doanduyhai.com/blog/?p=2325 Optionally I have pre-built some Zeppe

a solution of getting cassandra cross-datacenter latency at a certain time

2016-08-08 Thread Stone Fang
*objective*:get cassandra cross-datacenter latency in time *existing ticket:* there is a ticket [track cross-datacenter latency]( https://issues.apache.org/jira/browse/CASSANDRA-11569) but it is a statistics value from node starting,i want to get the instantaneous value in a certain time. *thoug