Re: Datacenter decommissioning on Cassandra 4.1.4

2024-04-22 Thread Alain Rodriguez via user
Hi Michalis, It's been a while since I removed a DC for the last time, but I see there is now a protection to avoid accidentally leaving a DC without auth capability. This was introduced in C* 4.1 through CASSANDRA-17478 ( https://issues.apache.org/jira/browse/CASSANDRA-17478). The process of dr

Re: Decommissioned nodes are in UNREACHABLE state

2020-05-25 Thread Alain RODRIGUEZ
t it would disappear on restart. Good luck with that. C*heers, --- Alain Rodriguez - alain.rodrig...@datastax.com France / Spain https://www.datastax.com Le sam. 23 mai 2020 à 19:07, Jai Bheemsen Rao Dhanwada < jaibheem...@gmail.com> a écrit : > any inputs here? &g

Re: Elevated response times from all nodes in a data center at the same time.

2019-10-16 Thread Alain RODRIGUEZ
ent logs (if you are having the issues NOW) - grep -e "WARN" -e "ERROR" /var/log/cassandra/system.log # to check what happened and was wrong For now I can't think about anything else, I hope some of those ideas will help you diagnose the problem. Once it is diagnosed, we sho

Re: cluster rolling restart

2019-10-16 Thread Alain RODRIGUEZ
arm (much), it might even be useful in some corner cases, but restarting the cluster regularly, for no reasons, is definitely not part of 'best practices' around Cassandra imho. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle -

Re: Constant blocking read repair for such a tiny table

2019-10-15 Thread Alain RODRIGUEZ
ning for a long time, with no changes. What about Cassandra? Any recent operations? I hope that with this information we might be able to understand better and finally be able to help. ------- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache

Re: Assassinate fails

2019-08-29 Thread Alain RODRIGUEZ
C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le ven. 16 août 2019 à 08:16, Alex a écrit : > Hello Alain, > > long time - I had to wait for a quiet week to try this.

Re: Difficulties after nodetool removenode

2019-07-04 Thread Alain RODRIGUEZ
ill there you can also re-run the 'nodetool removenode'. Data will be streamed again (to all nodes) and compacted in the future eventually. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thel

Re: Adding new DC with different version of Cassandra

2019-07-04 Thread Alain RODRIGUEZ
heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 1 juil. 2019 à 21:38, Rahul Reddy a écrit : > Thanks Jeff, > > We want to migrate to Apache 3.11.3 once entire cluster in apache w

Re: Restore from EBS onto different cluster

2019-07-04 Thread Alain RODRIGUEZ
nal ones, not to sure here) and cluster the name you like. - Start seed nodes (with explicit tokens = old tokens) - Start remaining nodes. I'm answering without support and might be missing a step or 2, but I have been doing this successfully already and these are the disti

Re: Get information about GC pause (Stop the world) via JMX, it's possible ?

2019-07-04 Thread Alain RODRIGUEZ
;gceasy' ¯\_(ツ)_/¯ ). C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le jeu. 27 juin 2019 à 21:52, Avinash Mandava a écrit : > Here's the metrics you want. D

Re: Write count vs Local Write Count

2019-07-04 Thread Alain RODRIGUEZ
t the coordinator level per table or keyspace. It's always per host for client request counts metrics. I hope I did not make it more confusing, let me know if I'm unclear or if it doesn't suit your own observations. Thanks for the questions, I could also rework my understanding here

Re: Decommissioned nodes are in UNREACHABLE state

2019-06-20 Thread Alain RODRIGUEZ
escribe' cluster at all, I would say immediately, but still appear in 'nodetool gossipinfo' with a 'left' or 'remove' status. I hope that helps and that one of the above will do the trick (I'd bet on the assassinate :)). Also sorry it took us a whi

Re: JanusGraph and Cassandra

2019-06-20 Thread Alain RODRIGUEZ
well will not be able to help you here. If you have questions around the metrics as such, we might be able to help you though :). C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com

Re: Cassandra Tombstone

2019-06-20 Thread Alain RODRIGUEZ
g the rest of the post might be useful while working out the design of the schema and queries, in particular if you plan to use deletes/TTLs. I hope that helps, C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting ht

Re: node re-start delays , busy Deleting mc-txn-compaction/ Adding log file replica

2019-06-20 Thread Alain RODRIGUEZ
Again, I'm not really familiar with those traces. I find traces and debug pretty useless (or even counter-productive) in 99% of the cases, so I don't use them much. Le jeu. 20 juin 2019 à 12:25, Alain RODRIGUEZ a écrit : > Hello Asad, > > >> I’m on environment with apach

Re: node re-start delays , busy Deleting mc-txn-compaction/ Adding log file replica

2019-06-20 Thread Alain RODRIGUEZ
utage)? To speed up things, when nodes are not crashing, under normal circumstances, use 'nodetool drain' as part of stopping the node, before stopping/killing the service/process. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle -

Re: How to query TTL on collections ?

2019-06-20 Thread Alain RODRIGUEZ
few, it would be querying the whole map again. I had to test it because I could not remember about this, and I think my observations are making sense. Sadly, there is no 'good' syntax for this query, it's just not permitted at all I would say. Sorry I have no better news for you :).

Re: Can I cancel a decommissioning procedure??

2019-06-05 Thread Alain RODRIGUEZ
gt; for the info! > > Cheers > > > Sent with ProtonMail <https://protonmail.com> Secure Email. > > ‐‐‐ Original Message ‐‐‐ > On Tuesday, June 4, 2019 10:59 PM, Alain RODRIGUEZ > wrote: > > > the issue is that the rest nodes in the cluster marked it

Re: Can I cancel a decommissioning procedure??

2019-06-04 Thread Alain RODRIGUEZ
back to something more serious, there are ways to limit the amount and possible scope of those, such as good practices, testing and automations. C*heers, ------- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelast

Re: Nodetool status load value

2019-05-29 Thread Alain RODRIGUEZ
Yes, with this proportions it is perfectly ok. Nodes have a similar dataset and I imagine queries are well distributed. The situation seems to be normal, at least nothing looking wrong in this `nodetool status` output I would say. C*heers, ------- Alain Rodriguez - al...@thelastpickle.

Re: Sstableloader

2019-05-29 Thread Alain RODRIGUEZ
Hello, I can't answer this question about the sstableloader (even though I think it should be ok). My understanding, even though I'm not really up to date with latest Datastax work, is that DSE uses a modified but compatible version of Cassandra, for everything that is not 'DSE feature' specifical

Re: Collecting Latency Metrics

2019-05-29 Thread Alain RODRIGUEZ
;re in, you can explore available mbeans and find the metrics available in 'org.apache.cassandra.[...]'. It's not ideal as you search 'manually' but it allowed me to find some metrics in the past or fix issues from the doc above. Out of curiosity, may I ask what backend you used for

Re: Can sstable corruption cause schema mismatch?

2019-05-29 Thread Alain RODRIGUEZ
ot fully open across all nodes? - Anything in the logs? Do you know what triggered this situation in the first place? C*heers, ------- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le mar. 28 mai

Re: Can sstable corruption cause schema mismatch?

2019-05-28 Thread Alain RODRIGUEZ
ay, stopping repair and restarting it when all nodes are probably allows you a better understanding/control of what's going on. Also, it reduces the load in time of troubles or maintenance, when the cluster is somewhat weaker. C*heers, --- Alain Rodriguez - al...@thelast

Re: Ideal duration for max_hint_window_in_ms

2019-05-27 Thread Alain RODRIGUEZ
you might use this on worst node at the worst time to see if any threads are stacking in the 'pending' state. Also check for 'blocked' and 'dropped' If you'd like some help with your 'main issue' first, we would need more details and context. Hope t

Re: schema for testing that has a lot of edge cases

2019-05-27 Thread Alain RODRIGUEZ
) including Cassandra servers, client and monitoring (Prometheus). You can have a look anyway, I think both projects might be of interest to reach your goal. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting

Re: Optimal Heap Size Cassandra Configuration

2019-05-21 Thread Alain RODRIGUEZ
ou're tired of playing with GC, you can apply changes everywhere and observe impact on the cluster (latency/dropped messages/CPU load...) Hope that helps and completes somewhat Elliott's excellent answer. --- Alain Rodriguez - al...@thelastpickle.com France / Spain

Re: Corrupted sstables

2019-05-10 Thread Alain RODRIGUEZ
s/nodetool/rebuild_index.html Hope this helps, C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le jeu. 9 mai 2019 à 17:29, Jason Wee a écrit : > maybe print out value into the

Re: Schema Management Best Practices

2019-05-10 Thread Alain RODRIGUEZ
nges do not seem to be as safe and efficient as most of Cassandra internals are nowadays (for mainstream features, not counting MVs, Indexes, triggers, etc). This common feature that is to make multiple changes to your data model quickly should be handled with care and understanding in Cassandra an

Re: Backup Restore

2019-04-26 Thread Alain RODRIGUEZ
or restore, nothing obliges you to act node by node. When using restore, it often means the service is off. Restoring all nodes at once is possible and a good thing to do imho. Hope that helps! C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pic

Re: Can we set the precision of decimal column

2019-04-25 Thread Alain RODRIGUEZ
y, no imprecise representation of your number for the machine anymore. I said 1 arbitrarily because I've never seen a system that would need more, but it can be 100 or 10 depending on how big the number you work with are and the precision you want to keep. C*heers, ------- Alain

Re: Assassinate fails

2019-04-05 Thread Alain RODRIGUEZ
make sure you remove the good line, ensure it works on one node with no harm. I shared my feeling and I would try this fix. But it's ultimately your responsibility and I won't be behind the machine when you'll fix it. None of us will. Good luck ! :) C*heers, --- A

Re: Assassinate fails

2019-04-04 Thread Alain RODRIGUEZ
ybe?) in the gossip information (normally as 'LEFT'). This should not create the issue in 'nodetool describecluster' though. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastp

Re: Five Questions for Cassandra Users

2019-04-02 Thread Alain RODRIGUEZ
ctually help me with cluster management. Thus I see the interest of some tools to help people managing their cluster. Other alternatives are fully managed Cassandra clusters services if that's of interest for you, using the mailing list (as you did), working with consultant is another option (b

Re: Best practices while designing backup storage system for big Cassandra cluster

2019-04-01 Thread Alain RODRIGUEZ
lex topic, I hope some of this is helpful to you. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le jeu. 28 mars 2019 à 11:24, manish khandelwal < manishkhandelwa...@gmail.com> a écrit : > H

Re: Cassandra collection tombstones

2019-01-28 Thread Alain RODRIGUEZ
. Using this form worked without creating tombstones if I remember correctly. I would guess this is because when you 'update' you accept that the previously set values in the map remain unchanged if they are not specified, that's why Cassandra doesn't have to 'clean' first.

Re: Cassandra collection tombstones

2019-01-25 Thread Alain RODRIGUEZ
not collection does not change how the compaction strategy operates. I faced this in the past. Operationally you can have things working, but it's hard and really pointless (it was in my case at least). I would definitely recommend changing the model to update parts of the map and never

Re: question about the gain of increasing the number of vnode

2019-01-21 Thread Alain RODRIGUEZ
0 nodes and 4 vnodes, expect 13.323193263 neighbors - With 300 nodes and 256 vnodes, expect 198.8200470802 neighbors - With 300 nodes and 16 vnodes, expect 54.8867183963 neighbors - With 300 nodes and 4 vnodes, expect 15.4137752052 neighbors Good reading :). C*heers, ---

Re: question about the gain of increasing the number of vnode

2019-01-21 Thread Alain RODRIGUEZ
There have been some discussion on this topic in this mailing list, including a paper from Netflix with the impact of vnodes. I could not find it quickly, but I invite you to check. To share some ideas: More vnodes: + Better balance between nodes + maximize the streaming throughput for operations

Re: How can I limit the non-heap memory for Cassandra

2019-01-18 Thread Alain RODRIGUEZ
onfiguration would probably induce (faster/better) responses from us :). I hope this still helps. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le jeu. 3 janv. 2019 à 00:

Re: Cassandra 2.1 bootstrap - No streaming progress from one node

2018-11-26 Thread Alain RODRIGUEZ
d upgrade soon ;-)) C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le mer. 7 nov. 2018 à 19:22, Durity, Sean R a écrit : > I would wipe the new node and bootstrap again. I do not know of an

Re: Exception when running sstableloader

2018-11-26 Thread Alain RODRIGUEZ
ld you try with increasing this as well: 'thrift_max_message_length_in_mb' and see what happens? Cheers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 5 nov. 2018 à 1

Re: Problem with hints

2018-11-26 Thread Alain RODRIGUEZ
u're still using a version of Cassandra storing hints in the system table, you might want to upgrade? C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le jeu. 1 nov. 2018 à 16:22,

Re: View slow queries in V3

2018-11-26 Thread Alain RODRIGUEZ
of Prometheus. People also use Grafana/Graphite and other standard tools. You might find nice dashboards there too. I hope this will help (if it's not too late :)). C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandr

Re: snapshot strategy?

2018-11-21 Thread Alain RODRIGUEZ
-script-to-delete-directories-older-than-n-days . I hope I answered closer to your expectations this time :) C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 5 nov. 2018 à 21

Re: snapshot strategy?

2018-11-05 Thread Alain RODRIGUEZ
Hello Lou, how do you clear the automatic ones (e.g. names unknown) without clearing > the named ones? > The option '-t' might be what you are looking for: 'nodetool clearsnapshot -t nameOfMySnapshot'. >From the documentation here: http://cassandra.apache.org/doc/latest/tools/nodetool/clearsnaps

Re: Scrub - autocompaction

2018-10-29 Thread Alain RODRIGUEZ
. If you believe there might be an issue with your version of Cassandra or want to make sure that 'nodetool scrub' behaves as described, reading the code and/or observing writ time of the files that are getting compacted (SCRUB) is a way to go. C*heers, ------- Al

Re: nodetool status and node maintenance

2018-10-26 Thread Alain RODRIGUEZ
your nodes to restart. This way the failure is not invisible, but ignored when only bouncing the nodes. As a side note, be aware that the 'nodetool status' only give a view of the cluster from a specific node, that can be completely wrong as well :). C*heers, -

Re: Insert from Select - CQL

2018-10-25 Thread Alain RODRIGUEZ
x27; statement: https://stackoverflow.com/questions/21363046/how-to-select-data-from-a-table-and-insert-into-another-table I'm not too sure what suits you the best. C*heers, ------- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache

Re: Cleanup cluster after expansion?

2018-10-25 Thread Alain RODRIGUEZ
creen and see if the command returned or not and you don't have to kill the command just after running it. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 22 oct.

Re: Upgrade to version 3

2018-10-18 Thread Alain RODRIGUEZ
already, it might be just fine for you as well. Yet you might want to explore this issue and maybe consider to wait for this patch to be released to reduce the risks (or apply this patch yourself). C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The

Re: Mview in cassandra

2018-10-17 Thread Alain RODRIGUEZ
oment. If this cluster would be under my responsibility I would probably consider redesign the schema. C*heers, --- Alain Rodriguez - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le mar. 16 oct. 2018 à 18:25, rajasekhar kommineni a

Re: Connections info

2018-10-05 Thread Alain RODRIGUEZ
ion in one place. You can also run similar commands on the clients (Apps) toI hope that helps. C*heers --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le ven. 5 oct. 2018 à 06:28, M

Re: Metrics matrix: migrate 2.1.x metrics to 2.2.x+

2018-10-05 Thread Alain RODRIGUEZ
ve existing ones for himself or/and suggest improvements to Datadog :). I hope I'll do some more of this work at some point in the future. Good luck, C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting

Re: Metrics matrix: migrate 2.1.x metrics to 2.2.x+

2018-10-02 Thread Alain RODRIGUEZ
layer that removes this complexity (if I remember well, we built those dashboards a while ago). C*heers ------- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 1 oct. 2018 à 19:

Re: Re: Re: how to configure the Token Allocation Algorithm

2018-10-01 Thread Alain RODRIGUEZ
Hello again :), I thought a little bit more about this question, and I was actually wondering if something like this would work: Imagine 3 node cluster, and create them using: For the 3 nodes: `num_token: 4` Node 1: `intial_token: -9223372036854775808, -4611686018427387905, -2, 461168601842738790

Re: unsubscribe

2018-10-01 Thread Alain RODRIGUEZ
Hello, You're still subscribed to this mailing list I am afraid :). In case you missed it: To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org C*heers. Le lun. 1 oct. 2018 à 08:04, Gabriel Lindeborg < gabriel.lindeb...

Re: how to configure the Token Allocation Algorithm

2018-10-01 Thread Alain RODRIGUEZ
27;allocate_tokens_for_keyspace' won't affect this seed node (already initialized) in any way Also, do not forget to have more nodes becoming 'seeds', either after bootstrap or just start a couple more of seeds after the first one for example. C*heers, ---

Re: Metrics matrix: migrate 2.1.x metrics to 2.2.x+

2018-10-01 Thread Alain RODRIGUEZ
le.ReadLatency.95percentile, 2, 3), 1, 7, 8, 9) to aliasByNode(averageSeriesWithWildcards(cassandra.$env.$dc.$host.org.apache.cassandra.metrics.Table.ReadLatency.$ks.$table.p95, 2, 3), 1, 8, 9, 10) C*heers, ------- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain T

Re: TTL tombstones in Cassandra using LCS are cretaed in the same level data TTLed data?

2018-09-27 Thread Alain RODRIGUEZ
Yet it's good to think about the design beforehand indeed, generally, it's good if you can rotate the partitions over time, not to reuse old partitions for example. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickl

Re: How default_time_to_live would delete rows without tombstones in Cassandra?

2018-09-27 Thread Alain RODRIGUEZ
sing TTL per insert? > Yes, exactly this, C*heers. --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le lun. 17 sept. 2018 à 14:58, Gabriel Giussi a écrit : > > From > https://docs.d

Re: Error during truncate: Cannot achieve consistency level ALL , how to fix it

2018-09-27 Thread Alain RODRIGUEZ
Hello Shyam, I think Jonathan understood the meaning of 'RF'. He is suggesting you to look at all your keyspaces strategy/RF for the system_auth table and make sure to use the right replication factor (and probably a NetworkTopologyStrategy). This might help: cqlsh -e "DESCRIBE KEYSPACE system_au

Re: Adding datacenter and data verification

2018-09-17 Thread Alain RODRIGUEZ
Hello Pradeep, It looks good to me and it's a cool runbook for you to follow and for others to reuse. To make sure that cassandra nodes in one datacenter can see the nodes of > the other datacenter, add the seed node of the new datacenter in any of the > old datacenter’s nodes and restart that no

Re: Default Single DataCenter -> Multi DataCenter

2018-09-14 Thread Alain RODRIGUEZ
. The procedure looks to be about right, there are probably some details we are missing while reading but that you will spot while running the test. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache

Re: node replacement failed

2018-09-14 Thread Alain RODRIGUEZ
, for the same reason of having ranges unavailable as well. I am still thinking about it, but before going deeper, is this still an issue for you at the moment? C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra

Re: Read timeouts when performing rolling restart

2018-09-14 Thread Alain RODRIGUEZ
to wait for the data to be flushed. It can be multiple things, but having an interactive view of the pending requests might lead you to the root cause of the issue. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassa

Re: Anticompaction causing significant increase in disk usage

2018-09-12 Thread Alain RODRIGUEZ
ation is relevant to help you fix this issue. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com Le mer. 12 sept. 2018 à 10:14, Martin Mačura a écrit : > Hi, >

Re: Read timeouts when performing rolling restart

2018-09-12 Thread Alain RODRIGUEZ
ue here)? 'grep -e "WARN" -e "ERROR" /var/log/cassandra/system.log' I hope one of those will help, let us know if you need help to interpret some of the outputs, C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain

Re: Default Single DataCenter -> Multi DataCenter

2018-09-10 Thread Alain RODRIGUEZ
d. If you are not confident or have doubts, you can share more about the context and post your exact plan, as I did years ago in the mail previously linked. People here should be able to confirm the process is ok before you move forward, giving you an extra confidence. C*heers,

Re: node replacement failed

2018-09-10 Thread Alain RODRIGUEZ
ff with RF=3 and quorum reads is the latency increase and the resource usage. Maybe is there a better approach, I am not too sure, but I think I would try option 1 first in any case. It's less destructive, less risky, no token range movements, no empty nodes available. I am not sure about li

Re: nodetool cleanup - compaction remaining time

2018-09-06 Thread Alain RODRIGUEZ
eu. 6 sept. 2018 à 11:06, Oleksandr Shulgin < oleksandr.shul...@zalando.de> a écrit : > On Thu, Sep 6, 2018 at 11:50 AM Alain RODRIGUEZ > wrote: > >> >> Be aware that this behavior happens when the compaction throughput is set >> to *0 *(unthrottled/unlim

Re: nodetool cleanup - compaction remaining time

2018-09-06 Thread Alain RODRIGUEZ
eanup type of compaction indeed and not due to some changes you could have made in the compaction throughput threshold. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com L

Re: commitlog content

2018-08-30 Thread Alain RODRIGUEZ
atest/configuration/cassandra_config_file.html#memtable-cleanup-threshold ). Also, the data in Cassandra is compacted and compressed. Over a short time period of test or if the data is small compared to the memory available and fits mostly in memory, I can imagine that what you describe can happen. C*

Re: dynamic_snitch=false, prioritisation/order or reads from replicas

2018-08-07 Thread Alain RODRIGUEZ
). But for sure I have seen this disabled giving some really nice improvement (as many others here as you mentioned). Sometimes it was not helpful, but I have never seen this change being really harmful though. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com Fra

Re: dynamic_snitch=false, prioritisation/order or reads from replicas

2018-08-06 Thread Alain RODRIGUEZ
k it's safer to run with 'dynamic_snitch: false'. Anyway, it's possible to test it on a canary node (or entire rack) and look at the p99 for read latencies for example :). This ticket is old, but was precisely on that topic: https://issues.apache.org/jira/browse/CASSANDRA-6908

Re: concurrent_compactors via JMX

2018-07-24 Thread Alain RODRIGUEZ
hows it was somehow useful to you. This is enough for me. C*heers --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-07-19 19:21 GMT+02:00 Riccardo Ferrari : > Alain, > > I really appreci

Re: concurrent_compactors via JMX

2018-07-19 Thread Alain RODRIGUEZ
" I should not answer messages before waking up fully... :) C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-07-19 14:07 GMT+01:00 Alain RODRIGUEZ : > Ah

Re: concurrent_compactors via JMX

2018-07-19 Thread Alain RODRIGUEZ
tting >>> a small number sstables (nodetool tablehistogram) then you probably >>> don't need to increase the compaction speed. >>> >>> Let's say that the compaction throughput is not often the cause of >>> stress during peak hours nor a direct way to make t

Re: apache cassandra development process and future

2018-07-18 Thread Alain RODRIGUEZ
neral consensus in the mailing list lately. For Scylladb and Datastax I don't know about the version to use. You should ask them directly. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting htt

Re: concurrent_compactors via JMX

2018-07-17 Thread Alain RODRIGUEZ
of my colleague Jon about GC tuning for Apache Cassandra interesting: http://thelastpickle.com/blog/2018/04/11/gc-tuning.html. GC pressure is a very common way to optimize a Cassandra cluster, to adapt it to your workload/hardware. C*heers, --- Alain Rodriguez - @arodream - al...@

Re: CPU Spike with Jmx_exporter

2018-07-10 Thread Alain RODRIGUEZ
to improve it on your side, I would open an issue or a discussion on prometheus side ( https://github.com/prometheus/jmx_exporter/issues maybe?). C*heers, ------- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consu

Re: Write Time of a Row in Multi DC Cassandra Cluster

2018-07-10 Thread Alain RODRIGUEZ
ave the row in a majority of the replicas before triggering the read. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-07-10 8:24 GMT+01:00 Simon Fontana Oscars

Re: Paging in Cassandra

2018-07-10 Thread Alain RODRIGUEZ
Hello, It sounds like a client/coding issue. People are working with distinct clients to connect to Cassandra. And it looks like there are not many 'spring-data-cassandra' users around ¯\_(ツ)_/¯. You could try giving a try there see if you have more luck: https://spring.io/questions. C*heers, A

Re: Debugging high coordinator latencies

2018-07-04 Thread Alain RODRIGUEZ
could probably let you know how GC is performing. What hardware are you using? C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-07-01 8:26 GMT+01:00 Tunay Gür : > Thanks f

Re: C* in multiple AWS AZ's

2018-07-03 Thread Alain RODRIGUEZ
cent operating system (such as Ubuntu 16.04) to have the latest drivers for NVMe. NVMe support in Ubuntu 14.04 AMI is not reliable. It might be absent or lead to data corruption. Make sure the OS in use works well with this hardware. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-instance-st

Re: JVM Heap erratic

2018-07-03 Thread Alain RODRIGUEZ
nsequence (and not a cause) of an issue. Due to pending requests, wide partitions, ongoing compactions, repairs or an intensive workload, GC can pressure can increase and mask another underlying, and root issue. You might want to check that the cluster is healthy other than GC, as a lot of distinct intern

Re: how to immediately delete tombstones

2018-06-04 Thread Alain RODRIGUEZ
onsiderations. I'm using cassandra on a single node. > I would not play with commit logs with a single-node setup. But I imagine it is not a production 'cluster' either. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Las

Re: how to immediately delete tombstones

2018-05-31 Thread Alain RODRIGUEZ
Radovan wrote a post with more information on this: http://thelastpickle.com/blog/2018/03/21/hinted-handoff-gc-grace-demystified.html Good luck with your tombstones, again, those are a bit tricky to handle sometimes ;-) C*heers, --- Alain Rodriguez - @arodream - al...@th

Re: cassandra concurrent read performance problem

2018-05-28 Thread Alain RODRIGUEZ
t within the read block size (buffer) you are using can also make reads more efficient (when data is compressed, the chunk size determines the buffer size. I hope, this helps. I am curious about that one, please let us know what you find out :). C*heers, --- Alain Rodriguez -

Re: Log application Queries

2018-05-25 Thread Alain RODRIGUEZ
Nate wrote a post about this exact topic. In case it is of some use: http://thelastpickle.com/blog/2016/02/10/locking-down-apache-cassandra-logging.html C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra

Re: Cassandra Monitoring tool

2018-05-25 Thread Alain RODRIGUEZ
hot Apache Cassandra. My opinion here is biased most definitely :). I liked the interfaces of Datadog (using regularly) and Sematext (old experience) the most. C*heers, ------- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra

Re: unsubscribe

2018-05-25 Thread Alain RODRIGUEZ
Hello Matthias, I don't think you really left :). Give this address a try instead: user-unsubscr...@cassandra.apache.org ;-) C*iao, 2018-05-24 23:37 GMT+01:00 Matthias Hübner : > Ciao >

Re: Reading Data from C* Cluster

2018-05-25 Thread Alain RODRIGUEZ
not to reach the timeout. But I would really try not to implement this and pick some client that do that already. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-0

Re: repair in C* 3.11.2 and anticompactions

2018-05-24 Thread Alain RODRIGUEZ
ng > anticompacted, it cannot go through validation compaction)." > > I hope this helps, C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-05-23 21:48

Re: Why nodetool cleanup should be run sequentially after node joined a cluster

2018-04-11 Thread Alain RODRIGUEZ
://thelastpickle.com/blog/2017/08/14/limiting-nodetool-parallel-threads.html . C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-04-11 6:04 GMT+01:00 Christophe Schmitz

Re: write latency on single partition table

2018-04-09 Thread Alain RODRIGUEZ
in and we will hopefully able to help with extra information on the part that is slow :). C*heers! --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-04-07 6:05 GMT+01:00 on

Re: Urgent Problem - Disk full

2018-04-04 Thread Alain RODRIGUEZ
ize is something you could consider monitoring as it is way easier to fix it when the disk is not completely full and it can be fixed preemptively. Usually, 50 to 20% of free disk is recommended depending on your use case. C*heers, --- Alain Rodriguez - @arodream - al...@thelastp

Re: datastax cassandra minimum hardware recommendation

2018-04-04 Thread Alain RODRIGUEZ
Benchmark your workload, test it. This would be the most accurate answer here given the details we have. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-04-04

Re: nodetool repair and compact

2018-04-02 Thread Alain RODRIGUEZ
able closely before making this statement and I definitely will. Yet, I believe the second point still holds though: 'With UDC, you can trigger the compaction of the sstables you want to remove the tombstones from, instead of compacting *all* the sstables for a given table.' C*heers, 2018-0

Re: nodetool repair and compact

2018-04-02 Thread Alain RODRIGUEZ
ion such as the sstables sizes and sstablemetadata to get the tombstone ratio. C*heers, --- Alain Rodriguez - @arodream - al...@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-04-02 14:55 GMT+01:00 James Shaw : > you may

  1   2   3   4   5   6   7   8   >