Re: Change STCS to TWCS

[wxn...@zjqunshuo.com]

Thanks Jeff. Will have a try using JMX on one node.
 
From: Jeff Jirsa
Date: 2017-12-29 15:57
To: user
Subject: Re: Change STCS to TWCS
It’s going to cause a lot of compactions - this is especially true with stcs 
where many of your sstables (especially the big ones) will overlap and be joined

Monitor free space (and stop compactions as needed), free memory (bloom filters 
during compaction will take a big chunk as you build), and of course cpu and IO 
- compaction touches just about everything 

You can test the operation impact by changing it on just one instance using JMX 
- compaction strategy can be set as a json string and it won’t change the 
cluster wide schema (or persist through reboot).


-- 
Jeff Jirsa


On Dec 28, 2017, at 11:40 PM, "wxn...@zjqunshuo.com"  
wrote:

Hi All,
My production cluster is running 2.2.8. It is used to store time series data 
with only insertion with TTL, no update and deletion. From the mail lists seems 
TWCS is more suitable than STCS for my use case. I'm thinking about changing 
STCS to TWCS in production. I have read the 
guide(http://thelastpickle.com/blog/2016/12/08/TWCS-part1.html) someone have 
posted.

The cluster info:
UN XX.XX.44.149 939.23 GB 256 25.8% 9180b7c9-fa0b-4bbe-bf62-64a599c01e58 rack1 
UN XX.XX.106.218 995.4 GB 256 26.0% e24d13e2-96cb-4e8c-9d94-22498ad67c85 rack1 
UN XX.XX.42.113 905.85 GB 256 23.8% 385ad28c-0f3f-415f-9e0a-7fe8bef97e17 rack1 
UN XX.XX.41.165 859.85 GB 256 23.1% 46f37f06-9c45-492d-bd25-6fef7f926e38 rack1 
UN XX.XX.106.210 1.15 TB 256 26.8% a31b6088-0cb2-40b4-ac22-aec718dbd035 rack1 
UN XX.XX.104.41 900.21 GB 256 23.6% db08f0d7-d71f-400a-85a6-1f637fa839ee rack1 
UN XX.XX.41.95 960.89 GB 256 26.3% cf80924b-885f-42fb-b8f8-f9e1946ec30a rack1 
UN XX.XX.103.239 919.14 GB 256 24.7% c3f883a8-3643-46a1-ac7a-ea1b1046b400 rack1

I plan to use "alter table" to switch STCS to TWCS in production. My concern is:
1. Does the switch have a big impact on cluster performance?
2. To ensure a smooth switch, what could I pay attention to?

Best Regards,
-Simon

Re: Rebuild to a new DC fails every time

[Martin Mačura]

Is this something that can be resolved by CASSANDRA-11841 ?

Thanks,

Martin

On Thu, Dec 21, 2017 at 3:02 PM, Martin Mačura  wrote:
> Hi all,
> we are trying to add a new datacenter to the existing cluster, but the
> 'nodetool rebuild' command always fails after a couple of hours.
>
> We're on Cassandra 3.9.
>
> Example 1:
>
> 172.24.16.169 INFO  [STREAM-IN-/172.25.16.125:55735] 2017-12-13
> 23:55:38,840 StreamResultFuture.java:174 - [Stream
> #b8faf130-e092-11e7-bab5-0d4fb7c90e72 ID#0] Prepare completed.
> Receiving 0 files(0.000KiB), sending 9844 files(885.587GiB)
> 172.25.16.125 INFO  [STREAM-IN-/172.24.16.169:7000] 2017-12-13
> 23:55:38,858 StreamResultFuture.java:174 - [Stream
> #b8faf130-e092-11e7-bab5-0d4fb7c90e72 ID#0] Prepare completed.
> Receiving 9844 files(885.587GiB), sending 0 files(0.000KiB)
>
> 172.24.16.169 ERROR [STREAM-IN-/172.25.16.125:55735] 2017-12-14
> 04:28:09,064 StreamSession.java:533 - [Stream
> #b8faf130-e092-11e7-bab5-0d4fb7c90e72] Streaming error occurred on
> session with peer 172.25.16.125
> 172.24.16.169 java.io.IOException: Connection reset by peer
>
> 172.24.16.169 ERROR [STREAM-OUT-/172.25.16.125:49412] 2017-12-14
> 07:26:26,832 StreamSession.java:533 - [Stream
> #b8faf130-e092-11e7-bab5-0d4fb7c90e72] Streaming error occurred on
> session with peer 172.25.16.125
> 172.24.16.169 java.lang.RuntimeException: Transfer of file
> -13d78e3f11e6a6cbe1698349da4d/mc-8659-big-Data.db
> already completed or aborted (perhaps session failed?).
> 172.25.16.125 ERROR [STREAM-OUT-/172.24.16.169:7000] 2017-12-14
> 07:26:50,004 StreamSession.java:533 - [Stream
> #b8faf130-e092-11e7-bab5-0d4fb7c90e72] Streaming error occurred on
> session with peer 172.24.16.169
> 172.25.16.125 java.io.IOException: Connection reset by peer
>
> Example 2:
>
> 172.24.16.169 INFO  [STREAM-IN-/172.25.16.125:35202] 2017-12-18
> 03:24:31,423 StreamResultFuture.java:174 - [Stream
> #95d36300-e3d4-11e7-a90b-2b89506ad2af ID#0] Prepare completed.
> Receiving 0 files(0.000KiB), sending 12312 files(895.973GiB)
> 172.25.16.125 INFO  [STREAM-IN-/172.24.16.169:7000] 2017-12-18
> 03:24:31,441 StreamResultFuture.java:174 - [Stream
> #95d36300-e3d4-11e7-a90b-2b89506ad2af ID#0] Prepare completed.
> Receiving 12312 files(895.973GiB), sending 0 files(0.000KiB)
>
> 172.24.16.169 ERROR [STREAM-IN-/172.25.16.125:35202] 2017-12-18
> 06:39:42,049 StreamSession.java:533 - [Stream
> #95d36300-e3d4-11e7-a90b-2b89506ad2af] Streaming error occurred on
> session with peer 172.25.16.125
> 172.24.16.169 java.io.IOException: Connection reset by peer
>
> 172.24.16.169 ERROR [STREAM-OUT-/172.25.16.125:42744] 2017-12-18
> 09:25:36,188 StreamSession.java:533 - [Stream
> #95d36300-e3d4-11e7-a90b-2b89506ad2af] Streaming error occurred on
> session with peer 172.25.16.125
> 172.24.16.169 java.lang.RuntimeException: Transfer of file
> -3b5782d08e4411e6842917253f111990/mc-152979-big-Data.db
> already completed or aborted (perhaps session failed?).
> 172.25.16.125 ERROR [STREAM-OUT-/172.24.16.169:7000] 2017-12-18
> 09:25:59,447 StreamSession.java:533 - [Stream
> #95d36300-e3d4-11e7-a90b-2b89506ad2af] Streaming error occurred on
> session with peer 172.24.16.169
> 172.25.16.125 java.io.IOException: Connection timed out
>
> Datacenter: PRIMARY
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID
> Rack
> UN  172.24.16.169  918.31 GiB  256  100.0%
> bc4a980b-cca6-4ca2-b32f-f8206d48e14c  RAC1
> UN  172.24.16.170  908.76 GiB  256  100.0%
> 37b2742e-c83a-4341-896f-09d244810e69  RAC1
> UN  172.24.16.171  908.44 GiB  256  100.0%
> 6dc2b9d8-75dd-48f8-858c-53b1af42e8fb  RAC1
> Datacenter: SECONDARY
> =
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  AddressLoad   Tokens   Owns (effective)  Host ID
> Rack
> UN  172.25.16.125  27.48 GiB  256  100.0%
> 1e1669eb-cfd2-4718-a073-558946a8c947  RAC2
> UN  172.25.16.124  28.24 GiB  256  100.0%
> 896d9894-10c8-4269-9476-5ddab3c8abe9  RAC2
>
> Any ideas?
>
> Thanks,
>
> Martin

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: How to get page id without transmitting data to client

[Andy Tolbert]

Hi Eunsu,

Unfortunately there is not really a way to do this that I'm aware of.  The
page id contains data indicating where to start reading the next set of
rows (such as partition and clustering information), and in order to get to
that position you have to actually read the data.

The driver does have an API for manually specifying the page id to use and
we've documented some strategies

for storing and reusing the page id later, but not sure if that helps for
your particular use case.

Thanks,
Andy

On Thu, Dec 28, 2017 at 9:11 PM, Eunsu Kim  wrote:

> Hello everybody,
>
> I am using the datastax Java driver (3.3.0).
>
> When query large amounts of data, we set the fetch size (1) and
> transmit the data to the browser on a page-by-page basis.
>
> I am wondering if I can get the page id without receiving the real rows
> from the cassandra to my server.
>
> I only need 100 in front of 100,000. But I want the next page to be
> 11th.
>
> If you have a good idea, please share it.
>
> Thank you.
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Upgrade from 1.2.x to 2.0.x, upgradesstables has doubled the size on disk?

[Dan Washusen]

Hi All,
We're taking advantage of the lull in traffic to go through a production
cluster upgrade from 1.2.x (latest) to 2.0.x (latest). We have three nodes
with a replication factor of three. I've noticed that the 'space used' has
almost doubled as a result of running 'nodetool upgradesstables'.

Anyone have any ideas? Is that to be expected?

For comparison, on a node (pre-upgrade):

> nodetool cfstats dp.s_evt
> Keyspace: dp
> Read Count: 190570567
> Read Latency: 2.6280611004164145 ms.
> Write Count: 46213651
> Write Latency: 0.08166790944519835 ms.
> Pending Tasks: 0
> Column Family: s_evt
> SSTable count: 8
> Space used (live): 36269415929
> Space used (total): 36274282945
> SSTable Compression Ratio: 0.2345030140572
> Number of Keys (estimate): 3213696
> Memtable Columns Count: 2934
> Memtable Data Size: 9561951
> Memtable Switch Count: 1974
> Read Count: 190570567
> Read Latency: 2.628 ms.
> Write Count: 46213651
> Write Latency: 0.082 ms.
> Pending Tasks: 0
> Bloom Filter False Positives: 1162636
> Bloom Filter False Ratio: 0.73869
> Bloom Filter Space Used: 4492256
> Compacted row minimum size: 373
> Compacted row maximum size: 1996099046
> Compacted row mean size: 63595
> Average live cells per slice (last five minutes): 11.0
> Average tombstones per slice (last five minutes): 0.0


And after upgrading and running 'upgradesstables' (different node):

> nodetool cfstats dp.s_evt
> Keyspace: dp
> Read Count: 1461617
> Read Latency: 4.9734411921864625 ms.
> Write Count: 359250
> Write Latency: 0.11328054279749478 ms.
> Pending Tasks: 0
> Table: s_evt
> SSTable count: 6
> Space used (live), bytes: 71266932602
> Space used (total), bytes: 71266932602
> Off heap memory used (total), bytes: 44853104
> SSTable Compression Ratio: 0.2387480210082192
> Number of keys (estimate): 3307776
> Memtable cell count: 603223
> Memtable data size, bytes: 121913569
> Memtable switch count: 9
> Local read count: 1461617
> Local read latency: 7.248 ms
> Local write count: 359250
> Local write latency: 0.110 ms
> Pending tasks: 0
> Bloom filter false positives: 2501
> Bloom filter false ratio: 0.01118
> Bloom filter space used, bytes: 4135248
> Bloom filter off heap memory used, bytes: 4135200
> Index summary off heap memory used, bytes: 723576
> Compression metadata off heap memory used, bytes: 39994328
> Compacted partition minimum bytes: 536
> Compacted partition maximum bytes: 2874382626
> Compacted partition mean bytes: 108773
> Average live cells per slice (last five minutes): 11.0
> Average tombstones per slice (last five minutes): 17.0


Column familiy definition:

> create column family s_evt with column_type = 'Super' and comparator =
> 'TimeUUIDType' and subcomparator = 'UTF8Type';



Also curious why the 'Average tombstones per slice' value has gone from 0
to 17. Note sure if its relevant but way back when we used to write values
to that (super) column family with a TTL, but for a long time now its been
append only (with no TTL)...

Thanks,
Dan