Issue after dropping a counter column from a table

2019-10-09 Thread Felipe Esteves
Hi,

Recently, I've added a new counter column (let's call it A) to a counter
table, C*3.11.4
I've had to remove it thereafter, because of an application issue.
Later, the recreation was needed.
When I've executed the ADD command again, got an error for recreating
counter column, so I've created it with a new name (let's call it B).

However, now I've got this issue: when I try to restore a snapshot backup
using sstableloader, the process fails because A doesn't exist in the data,
but appears in the schema.cql generated by the snapshot command:
CREATE TABLE IF NOT EXISTS keyspace.table(
id uuid PRIMARY KEY,
A counter,
B counter
)
.
ALTER TABLE keyspace.table DROP A USING TIMESTAMP 1569582510859000;

If I do a select in system_schema.dropped_columns, it shows the
dropped counter column.

How can I properly remove all references to it?


Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com 

-- 



Re: Huge number of Mismatchs in the logs

2019-01-30 Thread Felipe Esteves
Hi, I've got the same problem
Upgraded from 2.2.11 to 3.11.3 and since then the DEBUG logs shows this
messages.

Seems to me that after the repair the messages were reduced a little, but
it's still happening.
In my case is a 3-node cluster, RF3, reads and writes with LOCAL_QUORUM

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com 

Tel.: (21) 3504-7162 ramal 57162

Skype: felipe2esteves


Em sáb, 1 de set de 2018 às 01:11, Mun Dega  escreveu:

> I upgraded from 2.1 to 3.11 and started to get huge number of Digest
> mismatch errors in the DEBUG log.  I ran a full repair in the cluster but
> it did not seem to cut back on these.
>
> My consistency is QUORUM so my guess is this error is on reads but would
> this also affect my writes to all replicas if my RF is 3?
>
> DEBUG [ReadRepairStage:1788] 2018-09-01 03:40:41,737 ReadCallback.java:242
> - Digest mismatch: org.apache.cassandra.service.DigestMismatchException:
> Mismatch for key DecoratedKey(-1046384702160736076,
> 423042334144313738374438) (6df35a84b4f9c1109342c744b84eead9 vs
> ef096894f14a8e2db75ed28396c570d3) at
> org.apache.cassandra.service.DigestResolver.compareResponses(DigestResolver.java:92)
> ~[apache-cassandra-3.11.2.jar:3.11.2] at
> org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:233)
> ~[apache-cassandra-3.11.2.jar:3.11.2] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_51] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_51] at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.2.jar:3.11.2] at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$7/343563528.run(Unknown
> Source) [apache-cassandra-3.11.2.jar:3.11.2] at
> java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
>
> Ma
>
>
> --
>
> Esta mensagem pode conter informações confidenciais e somente o indivíduo
> ou entidade a quem foi destinada pode utilizá-la. A transmissão incorreta
> da mensagem não acarreta a perda de sua confidencialidade. Caso esta
> mensagem tenha sido recebida por engano, solicitamos que o fato seja
> comunicado ao remetente e que a mensagem seja eliminada de seu sistema
> imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar,
> revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> comunicação sujeito a monitoramento.
>
> This message may include confidential information and only the intended
> addresses have the right to use it as is, or any part of it. A wrong
> transmission does not break its confidentiality. If you've received it
> because of a mistake or erroneous transmission, please notify the sender
> and delete it from your system immediately. This communication environment
> is controlled and monitored.
>
> B2W Digital
>
>
>

-- 



Re: How do you monitoring Cassandra Cluster?

2018-06-18 Thread Felipe Esteves
Hi, everyone,

I'm running some tests to monitor Cassandra 3.x with jmx_exporter +
prometheus + grafana.
I've managed to config it all and use the dashboard
https://grafana.com/dashboards/5408

However, I still can't aggregate metrics from all my cluster, just nodes
individually.
Any tips on how to do that?

Also, Opscenter gives some datacenter aggregation, I think it comes from
nodetool as I didn't seen any metrics about that.
Anyone having success on that?

cheers!

Em qua, 28 de jun de 2017 às 19:43, Petrus Gomes 
escreveu:

> I'm using JMX+Prometheus and Grafana.
> JMX = https://github.com/prometheus/jmx_exporter
> Prometheus + Grafana = https://prometheus.io/docs/visualization/grafana/
>
> There are some dashboard examples like that:
> https://grafana.com/dashboards/371
> Looks good.
>
> Thanks,
> Petrus Silva
>
> On Wed, Jun 28, 2017 at 5:55 AM, Peng Xiao <2535...@qq.com> wrote:
>
>> Dear All,
>>
>> we are currently using Cassandra 2.1.13,and it has grown to 5TB size with
>> 32 nodes in one DC.
>> For monitoring,opsCenter does not  send alarm and not free in higher
>> version.so we have to use a simple JMX+Zabbix template.And we plan to use
>> Jolokia+JMX2Graphite to draw the metrics chart now.
>>
>> Could you please advise?
>>
>> Thanks,
>> Henry
>>
>
>
> --
>
> Esta mensagem pode conter informações confidenciais e somente o indivíduo
> ou entidade a quem foi destinada pode utilizá-la. A transmissão incorreta
> da mensagem não acarreta a perda de sua confidencialidade. Caso esta
> mensagem tenha sido recebida por engano, solicitamos que o fato seja
> comunicado ao remetente e que a mensagem seja eliminada de seu sistema
> imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar,
> revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> comunicação sujeito a monitoramento.
>
> This message may include confidential information and only the intended
> addresses have the right to use it as is, or any part of it. A wrong
> transmission does not break its confidentiality. If you've received it
> because of a mistake or erroneous transmission, please notify the sender
> and delete it from your system immediately. This communication environment
> is controlled and monitored.
>
> B2W Digital
>
>
>
-- 

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com 

Tel.: (21) 3504-7162 ramal 57162

-- 



Re: Migrate from DSE (Datastax) to Apache Cassandra

2017-08-16 Thread Felipe Esteves
Ioannis,
As some people already said, there's one or two keyspaces that uses
EverywhereStrategy, dse_system is one of them, if I'm not wrong.
You  must remember to change them to a community strategy or it will fail.

-- 



Re: Cassandra seems slow when having many read operations

2017-07-21 Thread Felipe Esteves
Hi, Petrus,

Seems we've solved the problem, but it wasn't relationed to repair the
cluster or disk latency.
I've increased the memory available for Cassandra from 16GB to 24GB and the
performance was much improved!
The main symptom we've observed in Opscenter was a significantly decrease
in total compactions graph.

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>



2017-07-15 3:23 GMT-03:00 Petrus Gomes <petru...@gmail.com>:

> Hi Felipe,
>
> Yes, try it and let us know how it goes.
>
> Thanks,
> Petrus Silva.
>
> On Fri, Jul 14, 2017 at 11:37 AM, Felipe Esteves <
> felipe.este...@b2wdigital.com> wrote:
>
>> Hi Petrus, thanks for the feedback.
>>
>> I couldn't found the percent repaired in nodetool info, C* version is
>> 2.1.8, maybe it's something newer than that?
>>
>> I'm analyzing this thread about num_token.
>>
>> Compaction is "compaction_throughput_mb_per_sec: 16", I don't get
>> pending compactions in Opscenter.
>>
>> One point I've noticed, is that Opscenter show "OS: Disk Latency" max
>> with high values when the problem occurs, but it doesn't reflect in server
>> directly monitoring, in these tools the IO and latency of disks seems ok.
>> But seems to me that "read repair attempted" is a bit high, maybe it will
>> explain the latency in reads. I will try to run a repair on cluster to see
>> how it goes.
>>
>> Felipe Esteves
>>
>> Tecnologia
>>
>> felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>
>>
>> Tel.: (21) 3504-7162 ramal 57162
>>
>> Skype: felipe2esteves
>>
>> 2017-07-13 15:02 GMT-03:00 Petrus Gomes <petru...@gmail.com>:
>>
>>> How is your Percent Repaired  when you run " nodetool info" ?
>>>
>>> Search for :
>>> "reduced num_token = improved performance ??" topic.
>>> The people were discussing that.
>>>
>>> How is your compaction is configured?
>>>
>>> Could you run the same process in command line to have a measurement?
>>>
>>> Thanks,
>>> Petrus Silva
>>>
>>>
>>>
>>> On Thu, Jul 13, 2017 at 7:49 AM, Felipe Esteves <
>>> felipe.este...@b2wdigital.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have a Cassandra 2.1 cluster running on AWS that receives high read
>>>> loads, jumping from 100k requests to 400k requests, for example. Then it
>>>> normalizes and later cames another high throughput.
>>>>
>>>> To the application, it appears that Cassandra is slow. However, cpu and
>>>> disk use is ok in every instance, row cache is enabled and with almost 100%
>>>> hit rate.
>>>>
>>>> The logs from Cassandra instances doesn't have any errors, nor
>>>> tombstone messages or something liked that. It's mostly compactions and
>>>> G1GC operations.
>>>>
>>>> Any hints on where to investigate more?
>>>>
>>>>
>>>> Felipe Esteves
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> Esta mensagem pode conter informações confidenciais e somente o
>>> indivíduo ou entidade a quem foi destinada pode utilizá-la. A transmissão
>>> incorreta da mensagem não acarreta a perda de sua confidencialidade. Caso
>>> esta mensagem tenha sido recebida por engano, solicitamos que o fato seja
>>> comunicado ao remetente e que a mensagem seja eliminada de seu sistema
>>> imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar,
>>> revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de
>>> comunicação sujeito a monitoramento.
>>>
>>> This message may include confidential information and only the intended
>>> addresses have the right to use it as is, or any part of it. A wrong
>>> transmission does not break its confidentiality. If you've received it
>>> because of a mistake or erroneous transmission, please notify the sender
>>> and delete it from your system immediately. This communication environment
>>> is controlled and monitored.
>>>
>>> B2W Digital
>>>
>>>
>>>
>>
>>
>>
>>
>

-- 



Re: Cassandra seems slow when having many read operations

2017-07-14 Thread Felipe Esteves
Hi Petrus, thanks for the feedback.

I couldn't found the percent repaired in nodetool info, C* version is
2.1.8, maybe it's something newer than that?

I'm analyzing this thread about num_token.

Compaction is "compaction_throughput_mb_per_sec: 16", I don't get pending
compactions in Opscenter.

One point I've noticed, is that Opscenter show "OS: Disk Latency" max with
high values when the problem occurs, but it doesn't reflect in server
directly monitoring, in these tools the IO and latency of disks seems ok.
But seems to me that "read repair attempted" is a bit high, maybe it will
explain the latency in reads. I will try to run a repair on cluster to see
how it goes.

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

Skype: felipe2esteves

2017-07-13 15:02 GMT-03:00 Petrus Gomes <petru...@gmail.com>:

> How is your Percent Repaired  when you run " nodetool info" ?
>
> Search for :
> "reduced num_token = improved performance ??" topic.
> The people were discussing that.
>
> How is your compaction is configured?
>
> Could you run the same process in command line to have a measurement?
>
> Thanks,
> Petrus Silva
>
>
>
> On Thu, Jul 13, 2017 at 7:49 AM, Felipe Esteves <
> felipe.este...@b2wdigital.com> wrote:
>
>> Hi,
>>
>> I have a Cassandra 2.1 cluster running on AWS that receives high read
>> loads, jumping from 100k requests to 400k requests, for example. Then it
>> normalizes and later cames another high throughput.
>>
>> To the application, it appears that Cassandra is slow. However, cpu and
>> disk use is ok in every instance, row cache is enabled and with almost 100%
>> hit rate.
>>
>> The logs from Cassandra instances doesn't have any errors, nor tombstone
>> messages or something liked that. It's mostly compactions and G1GC
>> operations.
>>
>> Any hints on where to investigate more?
>>
>>
>> Felipe Esteves
>>
>>
>>
>>
>>
>
> --
>
> Esta mensagem pode conter informações confidenciais e somente o indivíduo
> ou entidade a quem foi destinada pode utilizá-la. A transmissão incorreta
> da mensagem não acarreta a perda de sua confidencialidade. Caso esta
> mensagem tenha sido recebida por engano, solicitamos que o fato seja
> comunicado ao remetente e que a mensagem seja eliminada de seu sistema
> imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar,
> revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de
> comunicação sujeito a monitoramento.
>
> This message may include confidential information and only the intended
> addresses have the right to use it as is, or any part of it. A wrong
> transmission does not break its confidentiality. If you've received it
> because of a mistake or erroneous transmission, please notify the sender
> and delete it from your system immediately. This communication environment
> is controlled and monitored.
>
> B2W Digital
>
>
>

-- 



Cassandra seems slow when having many read operations

2017-07-13 Thread Felipe Esteves
Hi,

I have a Cassandra 2.1 cluster running on AWS that receives high read
loads, jumping from 100k requests to 400k requests, for example. Then it
normalizes and later cames another high throughput.

To the application, it appears that Cassandra is slow. However, cpu and
disk use is ok in every instance, row cache is enabled and with almost 100%
hit rate.

The logs from Cassandra instances doesn't have any errors, nor tombstone
messages or something liked that. It's mostly compactions and G1GC
operations.

Any hints on where to investigate more?


Felipe Esteves

-- 



User auth error during new DC rebuild

2016-11-11 Thread Felipe Esteves
Hi, I've got a strange behaviour when adding a new DC to my Cassandra
cluster.

Cassandra version is 2.1.8.

The topology is a 3 DC cluster (let's call it A, B, C)  and I have 3 users
(1 superuser, 2 non-superuser)
I'm adding a new DC (let's call it D) with 1 node, so I started it with
auto_bootstrap=false, changed the keyspaces replication to include the new
DC and ran "nodetool rebuild C" from the D node.

I've altered the application keyspace as well the system_auth replication.

However, 1 of the non-superusers is getting authorization error when I
connect to any node on C (the DC that is streaming the rebuild) or D (the
new DC).
If I connect to A or B, it works.
The superuser and the other non-superuser connects in all nodes.


Any tips on why this is happening? Doesn't make sense to me just one of the
users not working.

The error is:

code=0100 [Bad credentials] message="Username and/or password are incorrect"



Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

-- 



Re: Change authorization from AllowAllAuthorizer to CassandraAuthorizer

2016-06-08 Thread Felipe Esteves
Hi,

Just a feedback from my scenario, it all went well, no downtime. In my
case, I had authentication enabled from the beginning, just needed to
change the authorizer.

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

Skype: felipe2esteves

2016-06-08 9:05 GMT-03:00 <sean_r_dur...@homedepot.com>:

> Which Cassandra version? Most of my authentication-from-non-authentication
> experience is from Cassandra 1.1 – 2.0. After that, I just enable from the
> beginning.
>
>
>
> Sean Durity – Lead Cassandra Admin
>
> Big DATA Team
>
> MTC 2250
>
> For support, create a JIRA
> <https://portal.homedepot.com/sites/bigdata/Shared%20Documents/Jira%20Hadoop%20Support%20Workflow.pdf>
>
>
>
> *From:* Jai Bheemsen Rao Dhanwada [mailto:jaibheem...@gmail.com]
> *Sent:* Tuesday, June 07, 2016 8:31 PM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Change authorization from AllowAllAuthorizer to
> CassandraAuthorizer
>
>
>
> Hello Sean,
>
>
> Recently I tried to enable Authentication on a existing cluster, I have
> see the below behaviour. (Clients already have the credentials and 3 node
> C* cluster)
>
>
>
> cluster 1 - Enabled Authentication on node1 by adding iptable rules (so
> that client will not communicate to this node) and I was able to connect to
> cql with default user and create the required users.
>
>
>
> cluster 2- Enabled Authentication on node1 by adding iptable rules but the
> default user was not created and below are the logs.
>
>
>
> WARN  [NonPeriodicTasks:1] 2016-06-07 20:59:17,898
> PasswordAuthenticator.java:230 - PasswordAuthenticator skipped default user
> setup: some nodes were not ready
>
> WARN  [NonPeriodicTasks:1] 2016-06-07 20:59:28,007 Auth.java:241 - Skipped
> default superuser setup: some nodes were not ready
>
>
>
> Any idea why the behaviour is not consistent across the two clusters?
>
>
>
> P.S: In both the cases the *system_auth *keyspace was created when the
> first node was updated.
>
>
>
> On Tue, Jun 7, 2016 at 11:19 AM, Felipe Esteves <
> felipe.este...@b2wdigital.com> wrote:
>
> Thank you, Sean!
>
>
> *Felipe Esteves*
>
> Tecnologia
>
> felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>
>
> Tel.: (21) 3504-7162 ramal 57162
>
> Skype: felipe2esteves
>
>
>
> 2016-06-07 14:20 GMT-03:00 <sean_r_dur...@homedepot.com>:
>
> I answered a similar question here:
>
> https://groups.google.com/forum/#!topic/nosql-databases/lLBebUCjD8Y
>
>
>
>
>
> Sean Durity – Lead Cassandra Admin
>
>
>
> *From:* Felipe Esteves [mailto:felipe.este...@b2wdigital.com]
> *Sent:* Tuesday, June 07, 2016 12:07 PM
> *To:* user@cassandra.apache.org
> *Subject:* Change authorization from AllowAllAuthorizer to
> CassandraAuthorizer
>
>
>
> Hi guys,
>
>
>
> I have a Cassandra 2.1.8 Community cluster running with AllowAllAuthorizer
> and have to change it, so I can implement permissions in different users.
>
> As I've checked in the docs, seems like a simple change,
> from AllowAllAuthorizer to CassandraAuthorizer in cassandra.yaml.
>
> However, I'm a litte concerned about the performance of the cluster while
> I'm restarting all the nodes. Is it possible to have any downtime (access
> errors, maybe), as all the data was created with AllowAllAuthorizer?
>
> --
>
> *Felipe Esteves*
>
> Tecnologia
>
> felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>
>
> Tel.: (21) 3504-7162 ramal 57162
>
>
>
>
> --
>
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>
>

Re: Change authorization from AllowAllAuthorizer to CassandraAuthorizer

2016-06-07 Thread Felipe Esteves
Thank you, Sean!

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

Skype: felipe2esteves

2016-06-07 14:20 GMT-03:00 <sean_r_dur...@homedepot.com>:

> I answered a similar question here:
>
> https://groups.google.com/forum/#!topic/nosql-databases/lLBebUCjD8Y
>
>
>
>
>
> Sean Durity – Lead Cassandra Admin
>
>
>
> *From:* Felipe Esteves [mailto:felipe.este...@b2wdigital.com]
> *Sent:* Tuesday, June 07, 2016 12:07 PM
> *To:* user@cassandra.apache.org
> *Subject:* Change authorization from AllowAllAuthorizer to
> CassandraAuthorizer
>
>
>
> Hi guys,
>
>
>
> I have a Cassandra 2.1.8 Community cluster running with AllowAllAuthorizer
> and have to change it, so I can implement permissions in different users.
>
> As I've checked in the docs, seems like a simple change,
> from AllowAllAuthorizer to CassandraAuthorizer in cassandra.yaml.
>
> However, I'm a litte concerned about the performance of the cluster while
> I'm restarting all the nodes. Is it possible to have any downtime (access
> errors, maybe), as all the data was created with AllowAllAuthorizer?
>
> --
>
> *Felipe Esteves*
>
> Tecnologia
>
> felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>
>
> Tel.: (21) 3504-7162 ramal 57162
>
>
>
> --
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>

-- 



Change authorization from AllowAllAuthorizer to CassandraAuthorizer

2016-06-07 Thread Felipe Esteves
Hi guys,

I have a Cassandra 2.1.8 Community cluster running with AllowAllAuthorizer
and have to change it, so I can implement permissions in different users.
As I've checked in the docs, seems like a simple change,
from AllowAllAuthorizer to CassandraAuthorizer in cassandra.yaml.
However, I'm a litte concerned about the performance of the cluster while
I'm restarting all the nodes. Is it possible to have any downtime (access
errors, maybe), as all the data was created with AllowAllAuthorizer?
-- 

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

-- 



Re: Nodetool Rebuild sending few big packets of data. Is it normal?

2016-02-26 Thread Felipe Esteves
Hi Jeff,

Thanks for the info, you're right!

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

2016-02-26 17:38 GMT-03:00 Jeff Jirsa <jeff.ji...@crowdstrike.com>:

> Cassandra is streaming it at a near constant rate (if you had metrics for
> network interface, you’d probably see that), but it doesn’t register in
> nodetool status until it completes all of the sstables for a column family.
> At that point, the -tmp–Data.db files get renamed to drop the –tmp, and
> they become live on the node.
>
> I suspect you have a table/CF that’s approximately 47/48gb, and it
> completed, and it’s size in nodetool status jumped at that time.
>
>
>
> From: Felipe Esteves
> Reply-To: "user@cassandra.apache.org"
> Date: Friday, February 26, 2016 at 11:48 AM
> To: "user@cassandra.apache.org"
> Subject: Nodetool Rebuild sending few big packets of data. Is it normal?
>
> Hi,
>
> I'm running a nodetool rebuild to include a new DC in my cluster.
> My config is:
> DC1, 2 nodes per rack (2 racks), 70gb each node
> DC2, 2 nodes per rack (1 rack), 90gb each node
> DC3, 2 nodes per rack (1 rack) (*THIS IS THE NEW DC*)
>
> What I did was get the 2 nodes in DC3 up and running with bootstrap=false,
> and then ran a rebuild using DC2 as a parameter.
>
> However, when I started, the load in both new nodes rapidly increased to
> 1.4GB, according to nodetool status. And then it was slowly increasing for
> 4 hours, in a 10mb basis. Then, suddenly, 1 node had 49.5GB and the other
> followed soon.
> In the instance logs, I have only stream messages from when I've started
> the rebuild.
>
> My point is, is it normal to Cassandra accumulate this amount of data and
> then send it? I was hoping that it was more of a gradual and incremental
> proccess.
>
> thanks,
>
> Felipe Esteves
>
> Tecnologia
>
> felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>
>
> Tel.: (21) 3504-7162 ramal 57162
>
>
>
>
>

-- 



Nodetool Rebuild sending few big packets of data. Is it normal?

2016-02-26 Thread Felipe Esteves
Hi,

I'm running a nodetool rebuild to include a new DC in my cluster.
My config is:
DC1, 2 nodes per rack (2 racks), 70gb each node
DC2, 2 nodes per rack (1 rack), 90gb each node
DC3, 2 nodes per rack (1 rack) (*THIS IS THE NEW DC*)

What I did was get the 2 nodes in DC3 up and running with bootstrap=false,
and then ran a rebuild using DC2 as a parameter.

However, when I started, the load in both new nodes rapidly increased to
1.4GB, according to nodetool status. And then it was slowly increasing for
4 hours, in a 10mb basis. Then, suddenly, 1 node had 49.5GB and the other
followed soon.
In the instance logs, I have only stream messages from when I've started
the rebuild.

My point is, is it normal to Cassandra accumulate this amount of data and
then send it? I was hoping that it was more of a gradual and incremental
proccess.

thanks,

Felipe Esteves

Tecnologia

felipe.este...@b2wdigital.com <seu.em...@b2wdigital.com>

Tel.: (21) 3504-7162 ramal 57162

--