Re: very slow repair

[Laxmikant Upadhyay]

Few queries:
1. What is the cassandra version ?
2. is the size of table 4TB per node ?
3. What is the value of compaction_throughput_mb_per_sec and
stream_throughput_outbound_megabits_per_sec ?

On Thu, Jun 13, 2019 at 5:06 AM R. T. 
wrote:

> Hi,
>
> I am trying to run a repair for first time a specific column family in
> specific keyspace and it seems that is going super slow.
>
> I have 6 nodes cluster with 2 Datacenters (RF 2) and the repair is a non
> incremental, DC parallel one. This column family is around 4 TB and it is
> written heavily (compared with other CF) so since it is going to take 2
> months (according ETA in Reaper) does that mean that when this repair will
> finish the entropy will be again high in this CF ?
>
> How I can speed up the process ? Is there any way to diagnose bottlenecs?
>
> Thank you,
>
> W
>
>

-- 

regards,
Laxmikant Upadhyay

very slow repair

[R. T.]

Hi,

I am trying to run a repair for first time a specific column family in specific 
keyspace and it seems that is going super slow.

I have 6 nodes cluster with 2 Datacenters (RF 2) and the repair is a non 
incremental, DC parallel one. This column family is around 4 TB and it is 
written heavily (compared with other CF) so since it is going to take 2 months 
(according ETA in Reaper) does that mean that when this repair will finish the 
entropy will be again high in this CF ?

How I can speed up the process ? Is there any way to diagnose bottlenecs?

Thank you,

W

Decommissioned nodes are in UNREACHABLE state

[Jai Bheemsen Rao Dhanwada]

Hello,

I have a Cassandra cluster running with 2.1.16 version of Cassandra, where
I have decommissioned few nodes from the cluster using "nodetool
decommission", but I see the node IPs in UNREACHABLE state in "nodetool
describecluster" output. I believe  they appear only for 72 hours, but in
my case I see those nodes in UNREACHABLE for ever (more than 60 days).
Rolling restart of the nodes didn't remove them. any idea what could be
causing here?

Note: I don't see them in the nodetool status output.

Re: postmortem on 2.2.13 scale out difficulties

[Carl Mueller]

I posted a bug, cassandra-15155 :
https://issues.apache.org/jira/browse/CASSANDRA-15155?jql=project%20%3D%20CASSANDRA

It seems VERY similar to
https://issues.apache.org/jira/browse/CASSANDRA-6648

On Wed, Jun 12, 2019 at 12:14 PM Carl Mueller 
wrote:

> And once the cluster token map formation is done, it starts bootstrap and
> we get a ton of these:
>
> WARN  [MessagingService-Incoming-/2406:da14:95b:4503:910e:23fd:dafa:9983]
> 2019-06-12 15:22:04,760 IncomingTcpConnection.java:100 -
> UnknownColumnFamilyException reading from socket; closing
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
> cfId=df425400-c331-11e8-8b96-4b7f4d58af68
>
> And then after LOTS of those
>
> INFO  [main] 2019-06-12 15:23:25,515 StorageService.java:1142 - JOINING:
> Starting to bootstrap...
> INFO  [main] 2019-06-12 15:23:25,525 StreamResultFuture.java:87 - [Stream
> #05af9ee0-8d26-11e9-85c1-bd5476090c54] Executing streaming plan for
> Bootstrap
> INFO  [main] 2019-06-12 15:23:25,526 StorageService.java:1199 - Bootstrap
> completed! for the tokens [-7314981925085449175, ... bunch of tokens...
> 5499447097629838103]
>
>
> On Wed, Jun 12, 2019 at 12:07 PM Carl Mueller <
> carl.muel...@smartthings.com> wrote:
>
>> One node at a time: yes that is what we are doing
>>
>> We have not tried the streaming_socket_timeout_in_ms. It is currently 24
>> hours. (```streaming_socket_timeout_in_ms=8640```) which would cover
>> the bootstrap timeframe we have seen before (1-2 hours per node)
>>
>> Since it joins with no data, it is serving erroneous data. We may try
>> bootstrap rejoin and the JVM_OPT The node appears to think it has
>> bootstrapped even though the gossipinfo shows the new node has a different
>> schema version.
>>
>> We had scaled EU and US from 5 --> 25 without incident (one at a time),
>> and since we increased ring_delay_ms worked haphazardly to get us four
>> joins, and since then failure.
>>
>> The debug log shows:
>>
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
>> New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 9200286188287490229
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
>> New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 950856676715905899
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,563 MigrationManager.java:96 -
>> Not pulling schema because versions match or shouldPullSchemaFrom returned
>> false
>> INFO  [GossipStage:1] 2019-06-12 15:20:08,563 TokenMetadata.java:464 -
>> Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
>> INFO  [GossipStage:1] 2019-06-12 15:20:08,564 TokenMetadata.java:464 -
>> Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,565 MigrationManager.java:96 -
>> Not pulling schema because versions match or shouldPullSchemaFrom returned
>> false
>> INFO  [GossipStage:1] 2019-06-12 15:20:08,565 Gossiper.java:1027 - Node
>> /2600:1f18:4b4:5903:64af:955e:b65:8d83 is now part of the cluster
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,587 StorageService.java:1928 -
>> Node /2600:1f18:4b4:5903:64af:955e:b65:8d83 state NORMAL, token
>> [-1028768087263234868, .., 921670352349030554]
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
>> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
>> -1028768087263234868
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
>> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
>> -1045740236536355596
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
>> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
>> -1184422937682103096
>> DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
>> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
>> -1201924032068728250
>>
>> All the nodes appear to be reporting "Not pulling schema becuase versions
>> match or shouldPullSchmeaFrom returned false. That code
>> (MigrationManager.java) makes reference to a "gossip only" node, did we get
>> stuck in that somehow.
>>
>> On Wed, Jun 12, 2019 at 11:45 AM ZAIDI, ASAD A  wrote:
>>
>>>
>>>
>>>
>>>
>>> Adding one node at a time – is that successful?
>>>
>>> Check value of streaming_socket_timeout_in_ms parameter in
>>> cassandra.yaml and increase if needed.
>>>
>>> Have you tried Nodetool bootstrap resume & jvm option i.e.
>>> JVM_OPTS="$JVM_OPTS -Dcassandra.consistent.rangemovement=false"  ?
>>>
>>>
>>>
>>>
>>>
>>> *From:* Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID]
>>> *Sent:* Wednesday, June 12, 2019 11:35 AM
>>> *To:* user@cassandra.apache.org
>>> *Subject:* Re: postmortem on 2.2.13 scale out difficulties
>>>
>>>
>>>
>>> We only were able to scale out four nodes and then failures started
>>> occurring, including multiple instances of nodes joining a cluster without
>>> streaming.
>>>
>>>
>>>
>>> Sigh.
>>>
>>>
>>>
>>> On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller <
>>> 

Re: Recover lost node from backup or evict/re-add?

[Jon Haddad]

100% agree with Sean.  I would only use Cassandra backups in a case where
you need to restore from full cluster loss.  Example: An entire DC burns
down, tornado, flooding.

Your routine node replacement after a failure should be
replace_address_first_boot.

To ensure this goes smoothly, run regular repairs.  We (The Last Pickle)
maintain this to make it easy: http://cassandra-reaper.io/

Jon


On Wed, Jun 12, 2019 at 11:17 AM Durity, Sean R 
wrote:

> I’m not sure it is correct to say, “you cannot.” However, that is a more
> complicated restore and more likely to lead to inconsistent data and take
> longer to do. You are basically trying to start from a backup point and
> roll everything forward and catch up to current.
>
>
>
> Replacing/re-streaming is the well-trodden path. You are getting the net
> result of all that has happened since the node failure. And the node is not
> returning data to the clients while the bootstrap is running. If you have a
> restored/repairing node, it will accept client (and coordinator)
> connections even though it isn’t (guaranteed) consistent, yet.
>
>
>
> As I understand it – a full cluster recovery from backup still requires
> repair across the cluster to ensure consistency. In my experience, most
> apps cannot wait for a full restore/repair. Availability matters more. They
> also don’t want to pay for even more disk to hold some level of backups.
>
>
>
> There are some companies that provide finer-grained backup and recovery
> options, though.
>
>
>
> Sean Durity
>
>
>
> *From:* Alan Gano 
> *Sent:* Wednesday, June 12, 2019 1:43 PM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] RE: Recover lost node from backup or evict/re-add?
>
>
>
>
>
> Is it correct to say that a lost node cannot be restored from backup?  You
> must either replace the node or evict/re-add (i.e., rebuild from other
> nodes).
>
>
>
> Also, that snapshot, incremental, commitlog backups are relegated to
> application keyspace recovery only?
>
>
>
>
>
> How about recovery of the entire cluster? (rolling it back).  Are
> snapshots exact enough, in time, to not have a nodes that differ, in
> point-in-time, from the rest of the cluster?  Would those nodes be
> recoverable (nodetool repair?) … which brings me back to recovering a lost
> node from backup (restore last snapshot, and run nodetool repair?).
>
>
>
>
>
> Thanks,
>
>
>
> Alan Gano
>
>
>
>
>
> *From:* Jeff Jirsa [mailto:jji...@gmail.com ]
> *Sent:* Wednesday, June 12, 2019 10:14 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Recover lost node from backup or evict/re-add?
>
>
>
> A host can replace itself using the method I described
>
>
> On Jun 12, 2019, at 7:10 AM, Alan Gano  wrote:
>
> I guess I’m considering this scenario:
>
>- host and configuration have survived
>- /data is gone
>- /backups have survived
>
>
>
> I have tested recovering from this scenario with an evict/re-add, which
> worked fine.
>
>
>
> If I restore from backup, the node will be behind the cluster – e,
> does it get caught up after a restore and start it up?
>
>
>
> Alan
>
>
>
> *From:* Jeff Jirsa [mailto:jji...@gmail.com ]
> *Sent:* Wednesday, June 12, 2019 10:02 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Recover lost node from backup or evict/re-add?
>
>
>
> To avoid violating consistency guarantees, you have to repair the replicas
> while the lost node is down
>
>
>
> Once you do that it’s typically easiest to bootstrap a replacement
> (there’s a property named “replace address first boot” you can google or
> someone can link) that tells a new joining host to take over for a failed
> machine.
>
>
>
>
> On Jun 12, 2019, at 6:54 AM, Alan Gano  wrote:
>
>
>
> If I lose a node, does it make sense to even restore from
> snapshot/incrementals/commitlogs?
>
>
>
> Or is the best way to do an evict/re-add?
>
>
>
>
>
> Thanks,
>
>
>
> Alan.
>
>
>
> NOTICE: This communication is intended only for the person or entity to
> whom it is addressed and may contain confidential, proprietary, and/or
> privileged material. Unless you are the intended addressee, any review,
> reliance, dissemination, distribution, copying or use whatsoever of this
> communication is strictly prohibited. If you received this in error, please
> reply immediately and delete the material from all computers. Email sent
> through the Internet is not secure. Do not use email to send us
> confidential information such as credit card numbers, PIN numbers,
> passwords, Social Security Numbers, Account numbers, or other important and
> confidential information.
>
> NOTICE: This communication is intended only for the person or entity to
> whom it is addressed and may contain confidential, proprietary, and/or
> privileged material. Unless you are the intended addressee, any review,
> reliance, dissemination, distribution, copying or use whatsoever of this
> communication is strictly prohibited. If you received this in error, please
> reply immediately and delete the material 

RE: Recover lost node from backup or evict/re-add?

[Durity, Sean R]

I’m not sure it is correct to say, “you cannot.” However, that is a more 
complicated restore and more likely to lead to inconsistent data and take 
longer to do. You are basically trying to start from a backup point and roll 
everything forward and catch up to current.

Replacing/re-streaming is the well-trodden path. You are getting the net result 
of all that has happened since the node failure. And the node is not returning 
data to the clients while the bootstrap is running. If you have a 
restored/repairing node, it will accept client (and coordinator) connections 
even though it isn’t (guaranteed) consistent, yet.

As I understand it – a full cluster recovery from backup still requires repair 
across the cluster to ensure consistency. In my experience, most apps cannot 
wait for a full restore/repair. Availability matters more. They also don’t want 
to pay for even more disk to hold some level of backups.

There are some companies that provide finer-grained backup and recovery 
options, though.

Sean Durity

From: Alan Gano 
Sent: Wednesday, June 12, 2019 1:43 PM
To: user@cassandra.apache.org
Subject: [EXTERNAL] RE: Recover lost node from backup or evict/re-add?


Is it correct to say that a lost node cannot be restored from backup?  You must 
either replace the node or evict/re-add (i.e., rebuild from other nodes).

Also, that snapshot, incremental, commitlog backups are relegated to 
application keyspace recovery only?


How about recovery of the entire cluster? (rolling it back).  Are snapshots 
exact enough, in time, to not have a nodes that differ, in point-in-time, from 
the rest of the cluster?  Would those nodes be recoverable (nodetool repair?) … 
which brings me back to recovering a lost node from backup (restore last 
snapshot, and run nodetool repair?).


Thanks,

Alan Gano


From: Jeff Jirsa [mailto:jji...@gmail.com]
Sent: Wednesday, June 12, 2019 10:14 AM
To: user@cassandra.apache.org
Subject: Re: Recover lost node from backup or evict/re-add?

A host can replace itself using the method I described

On Jun 12, 2019, at 7:10 AM, Alan Gano mailto:ag...@tsys.com>> 
wrote:
I guess I’m considering this scenario:

  *   host and configuration have survived
  *   /data is gone
  *   /backups have survived

I have tested recovering from this scenario with an evict/re-add, which worked 
fine.

If I restore from backup, the node will be behind the cluster – e, does it 
get caught up after a restore and start it up?

Alan

From: Jeff Jirsa [mailto:jji...@gmail.com]
Sent: Wednesday, June 12, 2019 10:02 AM
To: user@cassandra.apache.org
Subject: Re: Recover lost node from backup or evict/re-add?

To avoid violating consistency guarantees, you have to repair the replicas 
while the lost node is down

Once you do that it’s typically easiest to bootstrap a replacement (there’s a 
property named “replace address first boot” you can google or someone can link) 
that tells a new joining host to take over for a failed machine.


On Jun 12, 2019, at 6:54 AM, Alan Gano mailto:ag...@tsys.com>> 
wrote:

If I lose a node, does it make sense to even restore from 
snapshot/incrementals/commitlogs?

Or is the best way to do an evict/re-add?


Thanks,

Alan.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.
NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.
NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, 

RE: Recover lost node from backup or evict/re-add?

[Alan Gano]

Is it correct to say that a lost node cannot be restored from backup?  You must 
either replace the node or evict/re-add (i.e., rebuild from other nodes).

Also, that snapshot, incremental, commitlog backups are relegated to 
application keyspace recovery only?


How about recovery of the entire cluster? (rolling it back).  Are snapshots 
exact enough, in time, to not have a nodes that differ, in point-in-time, from 
the rest of the cluster?  Would those nodes be recoverable (nodetool repair?) … 
which brings me back to recovering a lost node from backup (restore last 
snapshot, and run nodetool repair?).


Thanks,

Alan Gano


From: Jeff Jirsa [mailto:jji...@gmail.com]
Sent: Wednesday, June 12, 2019 10:14 AM
To: user@cassandra.apache.org
Subject: Re: Recover lost node from backup or evict/re-add?

A host can replace itself using the method I described

On Jun 12, 2019, at 7:10 AM, Alan Gano mailto:ag...@tsys.com>> 
wrote:
I guess I’m considering this scenario:

· host and configuration have survived

· /data is gone

· /backups have survived

I have tested recovering from this scenario with an evict/re-add, which worked 
fine.

If I restore from backup, the node will be behind the cluster – e, does it 
get caught up after a restore and start it up?

Alan

From: Jeff Jirsa [mailto:jji...@gmail.com]
Sent: Wednesday, June 12, 2019 10:02 AM
To: user@cassandra.apache.org
Subject: Re: Recover lost node from backup or evict/re-add?

To avoid violating consistency guarantees, you have to repair the replicas 
while the lost node is down

Once you do that it’s typically easiest to bootstrap a replacement (there’s a 
property named “replace address first boot” you can google or someone can link) 
that tells a new joining host to take over for a failed machine.


On Jun 12, 2019, at 6:54 AM, Alan Gano mailto:ag...@tsys.com>> 
wrote:

If I lose a node, does it make sense to even restore from 
snapshot/incrementals/commitlogs?

Or is the best way to do an evict/re-add?


Thanks,

Alan.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.
NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.

Re: postmortem on 2.2.13 scale out difficulties

[Carl Mueller]

And once the cluster token map formation is done, it starts bootstrap and
we get a ton of these:

WARN  [MessagingService-Incoming-/2406:da14:95b:4503:910e:23fd:dafa:9983]
2019-06-12 15:22:04,760 IncomingTcpConnection.java:100 -
UnknownColumnFamilyException reading from socket; closing
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
cfId=df425400-c331-11e8-8b96-4b7f4d58af68

And then after LOTS of those

INFO  [main] 2019-06-12 15:23:25,515 StorageService.java:1142 - JOINING:
Starting to bootstrap...
INFO  [main] 2019-06-12 15:23:25,525 StreamResultFuture.java:87 - [Stream
#05af9ee0-8d26-11e9-85c1-bd5476090c54] Executing streaming plan for
Bootstrap
INFO  [main] 2019-06-12 15:23:25,526 StorageService.java:1199 - Bootstrap
completed! for the tokens [-7314981925085449175, ... bunch of tokens...
5499447097629838103]


On Wed, Jun 12, 2019 at 12:07 PM Carl Mueller 
wrote:

> One node at a time: yes that is what we are doing
>
> We have not tried the streaming_socket_timeout_in_ms. It is currently 24
> hours. (```streaming_socket_timeout_in_ms=8640```) which would cover
> the bootstrap timeframe we have seen before (1-2 hours per node)
>
> Since it joins with no data, it is serving erroneous data. We may try
> bootstrap rejoin and the JVM_OPT The node appears to think it has
> bootstrapped even though the gossipinfo shows the new node has a different
> schema version.
>
> We had scaled EU and US from 5 --> 25 without incident (one at a time),
> and since we increased ring_delay_ms worked haphazardly to get us four
> joins, and since then failure.
>
> The debug log shows:
>
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
> New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 9200286188287490229
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
> New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 950856676715905899
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,563 MigrationManager.java:96 -
> Not pulling schema because versions match or shouldPullSchemaFrom returned
> false
> INFO  [GossipStage:1] 2019-06-12 15:20:08,563 TokenMetadata.java:464 -
> Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
> INFO  [GossipStage:1] 2019-06-12 15:20:08,564 TokenMetadata.java:464 -
> Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,565 MigrationManager.java:96 -
> Not pulling schema because versions match or shouldPullSchemaFrom returned
> false
> INFO  [GossipStage:1] 2019-06-12 15:20:08,565 Gossiper.java:1027 - Node
> /2600:1f18:4b4:5903:64af:955e:b65:8d83 is now part of the cluster
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,587 StorageService.java:1928 -
> Node /2600:1f18:4b4:5903:64af:955e:b65:8d83 state NORMAL, token
> [-1028768087263234868, .., 921670352349030554]
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
> -1028768087263234868
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
> -1045740236536355596
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
> -1184422937682103096
> DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
> New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
> -1201924032068728250
>
> All the nodes appear to be reporting "Not pulling schema becuase versions
> match or shouldPullSchmeaFrom returned false. That code
> (MigrationManager.java) makes reference to a "gossip only" node, did we get
> stuck in that somehow.
>
> On Wed, Jun 12, 2019 at 11:45 AM ZAIDI, ASAD A  wrote:
>
>>
>>
>>
>>
>> Adding one node at a time – is that successful?
>>
>> Check value of streaming_socket_timeout_in_ms parameter in cassandra.yaml
>> and increase if needed.
>>
>> Have you tried Nodetool bootstrap resume & jvm option i.e.
>> JVM_OPTS="$JVM_OPTS -Dcassandra.consistent.rangemovement=false"  ?
>>
>>
>>
>>
>>
>> *From:* Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID]
>> *Sent:* Wednesday, June 12, 2019 11:35 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* Re: postmortem on 2.2.13 scale out difficulties
>>
>>
>>
>> We only were able to scale out four nodes and then failures started
>> occurring, including multiple instances of nodes joining a cluster without
>> streaming.
>>
>>
>>
>> Sigh.
>>
>>
>>
>> On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller <
>> carl.muel...@smartthings.com> wrote:
>>
>> We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS,
>> IPV6
>>
>> Needed to scale out the asia datacenter, which was 5 nodes, europe and us
>> were 25 nodes
>>
>> We were running into bootstrapping issues where the new node failed to
>> bootstrap/stream, it failed with
>>
>>
>>
>> "java.lang.RuntimeException: A node required to move the data
>> consistently is 

Re: postmortem on 2.2.13 scale out difficulties

[Carl Mueller]

One node at a time: yes that is what we are doing

We have not tried the streaming_socket_timeout_in_ms. It is currently 24
hours. (```streaming_socket_timeout_in_ms=8640```) which would cover
the bootstrap timeframe we have seen before (1-2 hours per node)

Since it joins with no data, it is serving erroneous data. We may try
bootstrap rejoin and the JVM_OPT The node appears to think it has
bootstrapped even though the gossipinfo shows the new node has a different
schema version.

We had scaled EU and US from 5 --> 25 without incident (one at a time), and
since we increased ring_delay_ms worked haphazardly to get us four joins,
and since then failure.

The debug log shows:

DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 9200286188287490229
DEBUG [GossipStage:1] 2019-06-12 15:20:08,559 StorageService.java:1998 -
New node /2a05:d018:af:1108:86f4:d628:6bca:6983 at token 950856676715905899
DEBUG [GossipStage:1] 2019-06-12 15:20:08,563 MigrationManager.java:96 -
Not pulling schema because versions match or shouldPullSchemaFrom returned
false
INFO  [GossipStage:1] 2019-06-12 15:20:08,563 TokenMetadata.java:464 -
Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
INFO  [GossipStage:1] 2019-06-12 15:20:08,564 TokenMetadata.java:464 -
Updating topology for /2a05:d018:af:1108:86f4:d628:6bca:6983
DEBUG [GossipStage:1] 2019-06-12 15:20:08,565 MigrationManager.java:96 -
Not pulling schema because versions match or shouldPullSchemaFrom returned
false
INFO  [GossipStage:1] 2019-06-12 15:20:08,565 Gossiper.java:1027 - Node
/2600:1f18:4b4:5903:64af:955e:b65:8d83 is now part of the cluster
DEBUG [GossipStage:1] 2019-06-12 15:20:08,587 StorageService.java:1928 -
Node /2600:1f18:4b4:5903:64af:955e:b65:8d83 state NORMAL, token
[-1028768087263234868, .., 921670352349030554]
DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
-1028768087263234868
DEBUG [GossipStage:1] 2019-06-12 15:20:08,588 StorageService.java:1998 -
New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
-1045740236536355596
DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
-1184422937682103096
DEBUG [GossipStage:1] 2019-06-12 15:20:08,589 StorageService.java:1998 -
New node /2600:1f18:4b4:5903:64af:955e:b65:8d83 at token
-1201924032068728250

All the nodes appear to be reporting "Not pulling schema becuase versions
match or shouldPullSchmeaFrom returned false. That code
(MigrationManager.java) makes reference to a "gossip only" node, did we get
stuck in that somehow.

On Wed, Jun 12, 2019 at 11:45 AM ZAIDI, ASAD A  wrote:

>
>
>
>
> Adding one node at a time – is that successful?
>
> Check value of streaming_socket_timeout_in_ms parameter in cassandra.yaml
> and increase if needed.
>
> Have you tried Nodetool bootstrap resume & jvm option i.e.
> JVM_OPTS="$JVM_OPTS -Dcassandra.consistent.rangemovement=false"  ?
>
>
>
>
>
> *From:* Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID]
> *Sent:* Wednesday, June 12, 2019 11:35 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: postmortem on 2.2.13 scale out difficulties
>
>
>
> We only were able to scale out four nodes and then failures started
> occurring, including multiple instances of nodes joining a cluster without
> streaming.
>
>
>
> Sigh.
>
>
>
> On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller 
> wrote:
>
> We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS,
> IPV6
>
> Needed to scale out the asia datacenter, which was 5 nodes, europe and us
> were 25 nodes
>
> We were running into bootstrapping issues where the new node failed to
> bootstrap/stream, it failed with
>
>
>
> "java.lang.RuntimeException: A node required to move the data consistently
> is down"
>
>
>
> ...even though they were all up based on nodetool status prior to adding
> the node.
>
> First we increased the phi_convict_threshold to 12, and that did not help.
>
> CASSANDRA-12281 appeared similar to what we had problems with, but I don't
> think we hit that. Somewhere in there someone wrote
>
>
>
> "For us, the workaround is either deleting the data (then bootstrap
> again), or increasing the ring_delay_ms. And the larger the cluster is, the
> longer ring_delay_ms is needed. Based on our tests, for a 40 nodes cluster,
> it requires ring_delay_ms to be >50seconds. For a 70 nodes cluster,
> >100seconds. Default is 30seconds."
>
> Given the WAN nature or our DCs, we used ring_delay_ms to 100 seconds and
> it finally worked.
>
> side note:
>
> During the rolling restarts for setting phi_convict_threshold we observed
> quite a lot of status map variance between nodes (we have a program to poll
> all of a datacenter or cluster's view of the gossipinfo and statuses. AWS
> appears to have variance in networking based on the phi_convict_threshold
> advice, I'm 

RE: postmortem on 2.2.13 scale out difficulties

[ZAIDI, ASAD A]

Adding one node at a time – is that successful?
Check value of streaming_socket_timeout_in_ms parameter in cassandra.yaml and 
increase if needed.
Have you tried Nodetool bootstrap resume & jvm option i.e. JVM_OPTS="$JVM_OPTS 
-Dcassandra.consistent.rangemovement=false"  ?


From: Carl Mueller [mailto:carl.muel...@smartthings.com.INVALID]
Sent: Wednesday, June 12, 2019 11:35 AM
To: user@cassandra.apache.org
Subject: Re: postmortem on 2.2.13 scale out difficulties

We only were able to scale out four nodes and then failures started occurring, 
including multiple instances of nodes joining a cluster without streaming.

Sigh.

On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller 
mailto:carl.muel...@smartthings.com>> wrote:
We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS, IPV6

Needed to scale out the asia datacenter, which was 5 nodes, europe and us were 
25 nodes

We were running into bootstrapping issues where the new node failed to 
bootstrap/stream, it failed with

"java.lang.RuntimeException: A node required to move the data consistently is 
down"

...even though they were all up based on nodetool status prior to adding the 
node.

First we increased the phi_convict_threshold to 12, and that did not help.

CASSANDRA-12281 appeared similar to what we had problems with, but I don't 
think we hit that. Somewhere in there someone wrote

"For us, the workaround is either deleting the data (then bootstrap again), or 
increasing the ring_delay_ms. And the larger the cluster is, the longer 
ring_delay_ms is needed. Based on our tests, for a 40 nodes cluster, it 
requires ring_delay_ms to be >50seconds. For a 70 nodes cluster, >100seconds. 
Default is 30seconds."
Given the WAN nature or our DCs, we used ring_delay_ms to 100 seconds and it 
finally worked.

side note:

During the rolling restarts for setting phi_convict_threshold we observed quite 
a lot of status map variance between nodes (we have a program to poll all of a 
datacenter or cluster's view of the gossipinfo and statuses. AWS appears to 
have variance in networking based on the phi_convict_threshold advice, I'm not 
sure if our difficulties were typical in that regard and/or if our IPV6 and/or 
globally distributed datacenters were exacerbating factors.

We could not reproduce this in loadtest, although loadtest is only eu and us 
(but is IPV6)

Re: postmortem on 2.2.13 scale out difficulties

[Carl Mueller]

We're getting

DEBUG [GossipStage:1] 2019-06-12 15:20:07,797 MigrationManager.java:96 -
Not pulling schema because versions match or shouldPullSchemaFrom returned
false

multiple times, as it contacts the nodes.

On Wed, Jun 12, 2019 at 11:35 AM Carl Mueller 
wrote:

> We only were able to scale out four nodes and then failures started
> occurring, including multiple instances of nodes joining a cluster without
> streaming.
>
> Sigh.
>
> On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller 
> wrote:
>
>> We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS,
>> IPV6
>>
>> Needed to scale out the asia datacenter, which was 5 nodes, europe and us
>> were 25 nodes
>>
>> We were running into bootstrapping issues where the new node failed to
>> bootstrap/stream, it failed with
>>
>> "java.lang.RuntimeException: A node required to move the data
>> consistently is down"
>>
>> ...even though they were all up based on nodetool status prior to adding
>> the node.
>>
>> First we increased the phi_convict_threshold to 12, and that did not
>> help.
>>
>> CASSANDRA-12281 appeared similar to what we had problems with, but I
>> don't think we hit that. Somewhere in there someone wrote
>>
>> "For us, the workaround is either deleting the data (then bootstrap
>> again), or increasing the ring_delay_ms. And the larger the cluster is, the
>> longer ring_delay_ms is needed. Based on our tests, for a 40 nodes cluster,
>> it requires ring_delay_ms to be >50seconds. For a 70 nodes cluster,
>> >100seconds. Default is 30seconds."
>>
>> Given the WAN nature or our DCs, we used ring_delay_ms to 100 seconds and
>> it finally worked.
>>
>> side note:
>>
>> During the rolling restarts for setting phi_convict_threshold we observed
>> quite a lot of status map variance between nodes (we have a program to poll
>> all of a datacenter or cluster's view of the gossipinfo and statuses. AWS
>> appears to have variance in networking based on the phi_convict_threshold
>> advice, I'm not sure if our difficulties were typical in that regard and/or
>> if our IPV6 and/or globally distributed datacenters were exacerbating
>> factors.
>>
>> We could not reproduce this in loadtest, although loadtest is only eu and
>> us (but is IPV6)
>>
>

Re: postmortem on 2.2.13 scale out difficulties

[Carl Mueller]

We only were able to scale out four nodes and then failures started
occurring, including multiple instances of nodes joining a cluster without
streaming.

Sigh.

On Tue, Jun 11, 2019 at 3:11 PM Carl Mueller 
wrote:

> We had a three-DC (asia-tokyo/europe/us) cassandra 2.2.13 cluster, AWS,
> IPV6
>
> Needed to scale out the asia datacenter, which was 5 nodes, europe and us
> were 25 nodes
>
> We were running into bootstrapping issues where the new node failed to
> bootstrap/stream, it failed with
>
> "java.lang.RuntimeException: A node required to move the data consistently
> is down"
>
> ...even though they were all up based on nodetool status prior to adding
> the node.
>
> First we increased the phi_convict_threshold to 12, and that did not help.
>
> CASSANDRA-12281 appeared similar to what we had problems with, but I don't
> think we hit that. Somewhere in there someone wrote
>
> "For us, the workaround is either deleting the data (then bootstrap
> again), or increasing the ring_delay_ms. And the larger the cluster is, the
> longer ring_delay_ms is needed. Based on our tests, for a 40 nodes cluster,
> it requires ring_delay_ms to be >50seconds. For a 70 nodes cluster,
> >100seconds. Default is 30seconds."
>
> Given the WAN nature or our DCs, we used ring_delay_ms to 100 seconds and
> it finally worked.
>
> side note:
>
> During the rolling restarts for setting phi_convict_threshold we observed
> quite a lot of status map variance between nodes (we have a program to poll
> all of a datacenter or cluster's view of the gossipinfo and statuses. AWS
> appears to have variance in networking based on the phi_convict_threshold
> advice, I'm not sure if our difficulties were typical in that regard and/or
> if our IPV6 and/or globally distributed datacenters were exacerbating
> factors.
>
> We could not reproduce this in loadtest, although loadtest is only eu and
> us (but is IPV6)
>

ApacheCon North America 2019 Schedule Now Live!

[Rich Bowen]

Dear Apache Enthusiast,

(You’re receiving this message because you’re subscribed to one or more 
Apache Software Foundation project user mailing lists.)


We’re thrilled to announce the schedule for our upcoming conference, 
ApacheCon North America 2019, in Las Vegas, Nevada. See it now at 
https://www.apachecon.com/acna19/schedule.html  The event will be held 
September 9th through 12th at the Flamingo Hotel.  Register today at 
https://www.apachecon.com/acna19/register.html


Our schedule features keynotes by James Gosling, the father of Java; 
Samaira Mehta, founder and CEO of the Billion Kids Can Code project; and 
David Brin, noted science fiction author and futurist. And we’ll have a 
discussion panel featuring some of the founders of The Apache Software 
Foundation, talking about the past as well as their vision for the future.


ApacheCon is the flagship convention of the ASF, and features tracks 
curated by many of our project communities: Apache Drill, Apache Karaf, 
Big Data, TomcatCon, Apache Cloudstack, Integration, Apache Cassandra, 
Streaming, Geospatial software, Graph processing, Internet of Things, 
Community, Machine Learning, Apache Traffic Control, Apache Beam, 
Observability, OFBiz, and Mobile app development.


The Hackathon, which will run all day, every day, is the place to meet 
your project community, and get some serious work knocked out in a short 
focused time. The BarCamp is the place to discuss the topics that are 
important to you, with your colleagues, in an unconference format.


We offer financial assistance for travel and lodging for those who want 
to come to ApacheCon but are unable to afford it. Apply at 
http://apache.org/travel/ by June 21st to be considered for this.


If you’re unable to make it to North America, we’ll also be running 
ApacheCon Europe in Berlin in October. Details of that event are at 
https://aceu19.apachecon.com/


Follow us on Twitter - @ApacheCon - for all the latest updates. See you 
in Las Vegas!


Rich Bowen, VP Conferences, The ASF
rbo...@apache.org
http://apachecon.com/


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Recover lost node from backup or evict/re-add?

[Jeff Jirsa]

A host can replace itself using the method I described 

> On Jun 12, 2019, at 7:10 AM, Alan Gano  wrote:
> 
> I guess I’m considering this scenario:
> · host and configuration have survived
> · /data is gone
> · /backups have survived
>  
> I have tested recovering from this scenario with an evict/re-add, which 
> worked fine.
>  
> If I restore from backup, the node will be behind the cluster – e, does 
> it get caught up after a restore and start it up?
>  
> Alan
>  
> From: Jeff Jirsa [mailto:jji...@gmail.com] 
> Sent: Wednesday, June 12, 2019 10:02 AM
> To: user@cassandra.apache.org
> Subject: Re: Recover lost node from backup or evict/re-add?
>  
> To avoid violating consistency guarantees, you have to repair the replicas 
> while the lost node is down
>  
> Once you do that it’s typically easiest to bootstrap a replacement (there’s a 
> property named “replace address first boot” you can google or someone can 
> link) that tells a new joining host to take over for a failed machine.
>  
> 
> On Jun 12, 2019, at 6:54 AM, Alan Gano  wrote:
> 
>  
> If I lose a node, does it make sense to even restore from 
> snapshot/incrementals/commitlogs?
>  
> Or is the best way to do an evict/re-add?
>  
>  
> Thanks,
>  
> Alan.
>  
> NOTICE: This communication is intended only for the person or entity to whom 
> it is addressed and may contain confidential, proprietary, and/or privileged 
> material. Unless you are the intended addressee, any review, reliance, 
> dissemination, distribution, copying or use whatsoever of this communication 
> is strictly prohibited. If you received this in error, please reply 
> immediately and delete the material from all computers. Email sent through 
> the Internet is not secure. Do not use email to send us confidential 
> information such as credit card numbers, PIN numbers, passwords, Social 
> Security Numbers, Account numbers, or other important and confidential 
> information.
> NOTICE: This communication is intended only for the person or entity to whom 
> it is addressed and may contain confidential, proprietary, and/or privileged 
> material. Unless you are the intended addressee, any review, reliance, 
> dissemination, distribution, copying or use whatsoever of this communication 
> is strictly prohibited. If you received this in error, please reply 
> immediately and delete the material from all computers. Email sent through 
> the Internet is not secure. Do not use email to send us confidential 
> information such as credit card numbers, PIN numbers, passwords, Social 
> Security Numbers, Account numbers, or other important and confidential 
> information.

RE: Recover lost node from backup or evict/re-add?

[Alan Gano]

I guess I’m considering this scenario:

· host and configuration have survived

· /data is gone

· /backups have survived

I have tested recovering from this scenario with an evict/re-add, which worked 
fine.

If I restore from backup, the node will be behind the cluster – e, does it 
get caught up after a restore and start it up?

Alan

From: Jeff Jirsa [mailto:jji...@gmail.com]
Sent: Wednesday, June 12, 2019 10:02 AM
To: user@cassandra.apache.org
Subject: Re: Recover lost node from backup or evict/re-add?

To avoid violating consistency guarantees, you have to repair the replicas 
while the lost node is down

Once you do that it’s typically easiest to bootstrap a replacement (there’s a 
property named “replace address first boot” you can google or someone can link) 
that tells a new joining host to take over for a failed machine.


On Jun 12, 2019, at 6:54 AM, Alan Gano mailto:ag...@tsys.com>> 
wrote:

If I lose a node, does it make sense to even restore from 
snapshot/incrementals/commitlogs?

Or is the best way to do an evict/re-add?


Thanks,

Alan.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.

Re: Recover lost node from backup or evict/re-add?

[Jeff Jirsa]

To avoid violating consistency guarantees, you have to repair the replicas 
while the lost node is down

Once you do that it’s typically easiest to bootstrap a replacement (there’s a 
property named “replace address first boot” you can google or someone can link) 
that tells a new joining host to take over for a failed machine.


> On Jun 12, 2019, at 6:54 AM, Alan Gano  wrote:
> 
>  
> If I lose a node, does it make sense to even restore from 
> snapshot/incrementals/commitlogs?
>  
> Or is the best way to do an evict/re-add?
>  
>  
> Thanks,
>  
> Alan.
>  
> NOTICE: This communication is intended only for the person or entity to whom 
> it is addressed and may contain confidential, proprietary, and/or privileged 
> material. Unless you are the intended addressee, any review, reliance, 
> dissemination, distribution, copying or use whatsoever of this communication 
> is strictly prohibited. If you received this in error, please reply 
> immediately and delete the material from all computers. Email sent through 
> the Internet is not secure. Do not use email to send us confidential 
> information such as credit card numbers, PIN numbers, passwords, Social 
> Security Numbers, Account numbers, or other important and confidential 
> information.

Recover lost node from backup or evict/re-add?

[Alan Gano]

If I lose a node, does it make sense to even restore from 
snapshot/incrementals/commitlogs?

Or is the best way to do an evict/re-add?


Thanks,

Alan.

NOTICE: This communication is intended only for the person or entity to whom it 
is addressed and may contain confidential, proprietary, and/or privileged 
material. Unless you are the intended addressee, any review, reliance, 
dissemination, distribution, copying or use whatsoever of this communication is 
strictly prohibited. If you received this in error, please reply immediately 
and delete the material from all computers. Email sent through the Internet is 
not secure. Do not use email to send us confidential information such as credit 
card numbers, PIN numbers, passwords, Social Security Numbers, Account numbers, 
or other important and confidential information.

Re: Cassandra DataStax Java Driver in combination with Java EE / EJBs

[Ralph Soika]

Hi Stefan,
Hi John,

thanks for your answers, this helps me a lot.

@John: you are right, EJB does not bring any advantage in this case. I 
will change my classes to simple CDI.


I will write a short blog about this solution after I finished.

Best regards

Ralph

On 12.06.19 07:58, Stefan Miklosovic wrote:

Hi Ralph,

yes this is completely fine, even advisable. You can further extend
this idea to have sessions per keyspace for example if you really
insist, and it could be injectable based on some qualifier ... thats
up to you.

On Wed, 12 Jun 2019 at 11:31, John Sanda  wrote:

Hi Ralph,

A session is intended to be a long-lived, i.e., application-scoped object. You 
only need one session per cluster. I think what you are doing with the 
@Singleton is fine. In my opinion though, EJB really does not offer much value 
when working with Cassandra. I would be inclined to just use CDI.

Cheers

John

On Tue, Jun 11, 2019 at 5:38 PM Ralph Soika  wrote:

Hi,

I have a question concerning the Cassandra DataStax Java Driver in combination 
with Java EE and EJBs.

I have implemented a Rest Service API based on Java EE8. In my application I 
have for example a jax-rs rest resource to write data into cassandra cluster. 
My first approach was to create in each method call

  a new Casssandra Cluster and Session object,
  write my data into cassandra
  and finally close the session and the cluster object.

This works but it takes a lot of time (2-3 seconds) until the cluster object / 
session is opened for each request.

  So my second approach is now a @Singleton EJB providing the session object 
for my jax-rs resources. My service implementation to hold the Session object 
looks something like this:


@Singleton
public class ClusterService {
 private Cluster cluster;
 private Session session;

 @PostConstruct
 private void init() throws ArchiveException {
 cluster=initCluster();
 session = initArchiveSession();
 }

 @PreDestroy
 private void tearDown() throws ArchiveException {
 // close session and cluster object
 if (session != null) {
 session.close();
 }
 if (cluster != null) {
 cluster.close();
 }
 }

 public Session getSession() {
 if (session==null) {
 try {
 init();
 } catch (ArchiveException e) {
 logger.warning("unable to get falid session: " + 
e.getMessage());
 e.printStackTrace();
 }
 }
 return session;
 }

.

}


And my rest service calls now looking like this:


@Path("/archive")
@Stateless
public class ArchiveRestService {

 @EJB
 ClusterService clusterService;

 @POST
 @Consumes({ MediaType.APPLICATION_XML, MediaType.TEXT_XML })
 public Response postData(XMLDocument xmlDocument) {
 Session session = clusterService.getSession();
 session.execute();
 ...
 }
 ...
}


The result is now a super-fast behavior! Seems to be clear because my rest 
service no longer need to open a new session for each request.

My question is: Is this approach with a @Singleton ClusterService EJB valid or 
is there something I should avoid?
As far as I can see this works pretty fine and is really fast. I am running the 
application on a Wildfly 15 server which is Java EE8.

Thanks for your comments

Ralph




--

Imixs Software Solutions GmbH
Web: www.imixs.com Phone: +49 (0)89-452136 16
Office: Agnes-Pockels-Bogen 1, 80992 München
Registergericht: Amtsgericht Muenchen, HRB 136045
Geschaeftsführer: Gaby Heinle u. Ralph Soika

Imixs is an open source company, read more: www.imixs.org



--

- John

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org


--

*Imixs Software Solutions GmbH*
*Web:* www.imixs.com  *Phone:* +49 (0)89-452136 16
*Office:* Agnes-Pockels-Bogen 1, 80992 München
Registergericht: Amtsgericht Muenchen, HRB 136045
Geschaeftsführer: Gaby Heinle u. Ralph Soika

*Imixs* is an open source company, read more: www.imixs.org 

JanusGraph and Cassandra

[Vinayak Bali]

Hi,

I am using JanusGraph along with Cassandra as the backend. I have created a
graph using JanusGraphFactory.open() method. I want to create one more
graph and traverse it. Can you please help me.

Regards,
Vinayak