[jira] [Commented] (CASSANDRA-2356) make the debian package never start by default
[ https://issues.apache.org/jira/browse/CASSANDRA-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501723#comment-13501723 ] Tamar Fraenkel commented on CASSANDRA-2356: --- I would also like that fix. The auto restart is a pain after upgrading, since I need to merge my cassandra.yaml and cassander-env.sh, and don't want the node to restart on its own. > make the debian package never start by default > -- > > Key: CASSANDRA-2356 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2356 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Jeremy Hanna >Priority: Minor > Labels: debian, packaging > Attachments: 2356.txt > > > Currently the debian package that installs cassandra starts cassandra by > default. It sounds like that is a standard debian packaging convention. > However, if you want to bootstrap a new node and want to configure it before > it creates any sort of state information, it's a pain. I would think that > the common use case would be to have it install all of the init scripts and > such but *not* have it start up by default. That way an admin can configure > cassandra with seed, token, host, etc. information and then start it. That > makes it easier to programmatically do this as well - have chef/puppet > install cassandra, do some configuration, then do the service start. > With the current setup, it sounds like cassandra creates state on startup > that has to be cleaned before a new configuration can take effect. So the > process of installing turns into: > * install debian package > * shutdown cassandra > * clean out state (data/log dirs) > * configure cassandra > * start cassandra > That seems suboptimal for the default case, especially when trying to > automate new nodes being bootstrapped. > Another case might be when a downed node comes back up and starts by default > and tries to claim a token that has already been claimed by another newly > bootstrapped node. Rob is more familiar with that case so I'll let him > explain it in the comments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4446) nodetool drain sometimes doesn't mark commitlog fully flushed
[
https://issues.apache.org/jira/browse/CASSANDRA-4446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501722#comment-13501722
]
Tamar Fraenkel commented on CASSANDRA-4446:
---
I had the same experience, when I upgraded my cluster from 1.0.9 to 1.0.11. I
ran drain before the upgrade, upgrade on the node finished and node restarted
at 2012-11-20 10:20:58, but then I see in the logs reply of commit log:
{quote}
INFO [main] 2012-11-20 09:41:13,918 CommitLog.java (line 172) Replaying
/raid0/cassandra/commitlog/CommitLog-1353402218337.log
INFO [main] 2012-11-20 09:41:20,360 CommitLog.java (line 179) Log replay
complete, 0 replayed mutations
INFO [main] 2012-11-20 10:11:35,635 CommitLog.java (line 167) No commitlog
files found; skipping replay
INFO [main] 2012-11-20 10:21:11,631 CommitLog.java (line 172) Replaying
/raid0/cassandra/commitlog/CommitLog-1353404473899.log
INFO [main] 2012-11-20 10:21:18,119 CommitLog.java (line 179) Log replay
complete, 6413 replayed mutations
INFO [main] 2012-11-20 10:55:46,435 CommitLog.java (line 172) Replaying
/raid0/cassandra/commitlog/CommitLog-1353406871619.log
INFO [main] 2012-11-20 10:55:54,139 CommitLog.java (line 179) Log replay
complete, 3 replayed mutations
{quote}
This caused over increment of counters
> nodetool drain sometimes doesn't mark commitlog fully flushed
> -
>
> Key: CASSANDRA-4446
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4446
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.0.10
> Environment: ubuntu 10.04 64bit
> Linux HOSTNAME 2.6.32-345-ec2 #48-Ubuntu SMP Wed May 2 19:29:55 UTC 2012
> x86_64 GNU/Linux
> sun JVM
> cassandra 1.0.10 installed from apache deb
>Reporter: Robert Coli
> Attachments:
> cassandra.1.0.10.replaying.log.after.exception.during.drain.txt
>
>
> I recently wiped a customer's QA cluster. I drained each node and verified
> that they were drained. When I restarted the nodes, I saw the commitlog
> replay create a memtable and then flush it. I have attached a sanitized log
> snippet from a representative node at the time.
> It appears to show the following :
> 1) Drain begins
> 2) Drain triggers flush
> 3) Flush triggers compaction
> 4) StorageService logs DRAINED message
> 5) compaction thread excepts
> 6) on restart, same CF creates a memtable
> 7) and then flushes it [1]
> The columnfamily involved in the replay in 7) is the CF for which the
> compaction thread excepted in 5). This seems to suggest a timing issue
> whereby the exception in 5) prevents the flush in 3) from marking all the
> segments flushed, causing them to replay after restart.
> In case it might be relevant, I did an online change of compaction strategy
> from Leveled to SizeTiered during the uptime period preceding this drain.
> [1] Isn't commitlog replay not supposed to automatically trigger a flush in
> modern cassandra?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3736) -Dreplace_token leaves old node (IP) in the gossip with the token.
[
https://issues.apache.org/jira/browse/CASSANDRA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496136#comment-13496136
]
Tamar Fraenkel edited comment on CASSANDRA-3736 at 11/13/12 12:32 PM:
--
I have the same issue with Cassandra 1.0.11 (used DataStax AMI).
I thought it was supposed to be solved already.
I see those messages on the node that was started using
-Dcassandra.replace_token=.
>From time to time I also see
{color:blue}
INFO [GossipTasks:1] 2012-11-13 12:26:38,195 Gossiper.java (line 818)
InetAddress / is now dead.
INFO [GossipTasks:1] 2012-11-13 12:26:58,203 Gossiper.java (line 632)
FatClient / has been silent for 3ms, removing from gossip
INFO [GossipStage:1] 2012-11-13 12:27:59,210 Gossiper.java (line 838) Node
/ is now part of the cluster
INFO [GossipStage:1] 2012-11-13 12:27:59,210 Gossiper.java (line 804)
InetAddress / is now UP
INFO [GossipStage:1] 2012-11-13 12:27:59,210 StorageService.java (line 1017)
Nodes / and / have the same token
113427455640312821154458202477256070484. Ignoring /
{color}
was (Author: tamarfraenkel):
I have the same issue with Cassandra 1.0.11 (used DataStax AMI).
I thought it was supposed to be solved already.
I see those messages on the node that was started using
-Dcassandra.replace_token=.
> -Dreplace_token leaves old node (IP) in the gossip with the token.
> --
>
> Key: CASSANDRA-3736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.0.0
>Reporter: Jackson Chung
>Assignee: Vijay
> Fix For: 1.0.8, 1.1.0
>
> Attachments: 0001-CASSANDRA-3736.patch
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-957 introduce a
> -Dreplace_token,
> however, the replaced IP keeps on showing up in the Gossiper when starting
> the replacement node:
> {noformat}
> INFO [Thread-2] 2012-01-12 23:59:35,162 CassandraDaemon.java (line 213)
> Listening for thrift clients...
> INFO [GossipStage:1] 2012-01-12 23:59:35,173 Gossiper.java (line 836) Node
> /50.56.59.68 has restarted, now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,174 Gossiper.java (line 804)
> InetAddress /50.56.59.68 is now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,175 StorageService.java (line 988)
> Node /50.56.59.68 state jump to normal
> INFO [GossipStage:1] 2012-01-12 23:59:35,176 Gossiper.java (line 836) Node
> /50.56.58.55 has restarted, now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,176 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,177 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-12 23:59:45,048 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipTasks:1] 2012-01-13 00:00:06,062 Gossiper.java (line 632)
> FatClient /50.56.58.55 has been silent for 3ms, removing from gossip
> INFO [GossipStage:1] 2012-01-13 00:01:06,320 Gossiper.java (line 838) Node
> /50.56.58.55 is now part of the cluster
> INFO [GossipStage:1] 2012-01-13 00:01:06,320 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-13 00:01:06,321 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-13 00:01:16,106 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipTasks:1] 2012-01-13 00:01:37,121 Gossiper.java (line 632)
> FatClient /50.56.58.55 has been silent for 3ms, removing from gossip
> INFO [GossipStage:1] 2012-01-13 00:02:37,352 Gossiper.java (line 838) Node
> /50.56.58.55 is now part of the cluster
> INFO [GossipStage:1] 2012-01-13 00:02:37,353 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-13 00:02:37,353 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-13 00:02:47,158 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipStage:1] 2012-01-13 00:02:50,162 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipStage:1] 2012-01-13 00:02:50,163 StorageService.java (line 1156)
> Removing token 122029383590318827259508597176866581733 for /50.56.58.55
> {noformat}
> in the above, /50.56.58.55 was the replaced IP.
> tried adding the "Gossiper.instance.r
[jira] [Commented] (CASSANDRA-3736) -Dreplace_token leaves old node (IP) in the gossip with the token.
[
https://issues.apache.org/jira/browse/CASSANDRA-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496136#comment-13496136
]
Tamar Fraenkel commented on CASSANDRA-3736:
---
I have the same issue with Cassandra 1.0.11 (used DataStax AMI).
I thought it was supposed to be solved already.
I see those messages on the node that was started using
-Dcassandra.replace_token=.
> -Dreplace_token leaves old node (IP) in the gossip with the token.
> --
>
> Key: CASSANDRA-3736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.0.0
>Reporter: Jackson Chung
>Assignee: Vijay
> Fix For: 1.0.8, 1.1.0
>
> Attachments: 0001-CASSANDRA-3736.patch
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-957 introduce a
> -Dreplace_token,
> however, the replaced IP keeps on showing up in the Gossiper when starting
> the replacement node:
> {noformat}
> INFO [Thread-2] 2012-01-12 23:59:35,162 CassandraDaemon.java (line 213)
> Listening for thrift clients...
> INFO [GossipStage:1] 2012-01-12 23:59:35,173 Gossiper.java (line 836) Node
> /50.56.59.68 has restarted, now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,174 Gossiper.java (line 804)
> InetAddress /50.56.59.68 is now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,175 StorageService.java (line 988)
> Node /50.56.59.68 state jump to normal
> INFO [GossipStage:1] 2012-01-12 23:59:35,176 Gossiper.java (line 836) Node
> /50.56.58.55 has restarted, now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,176 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-12 23:59:35,177 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-12 23:59:45,048 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipTasks:1] 2012-01-13 00:00:06,062 Gossiper.java (line 632)
> FatClient /50.56.58.55 has been silent for 3ms, removing from gossip
> INFO [GossipStage:1] 2012-01-13 00:01:06,320 Gossiper.java (line 838) Node
> /50.56.58.55 is now part of the cluster
> INFO [GossipStage:1] 2012-01-13 00:01:06,320 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-13 00:01:06,321 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-13 00:01:16,106 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipTasks:1] 2012-01-13 00:01:37,121 Gossiper.java (line 632)
> FatClient /50.56.58.55 has been silent for 3ms, removing from gossip
> INFO [GossipStage:1] 2012-01-13 00:02:37,352 Gossiper.java (line 838) Node
> /50.56.58.55 is now part of the cluster
> INFO [GossipStage:1] 2012-01-13 00:02:37,353 Gossiper.java (line 804)
> InetAddress /50.56.58.55 is now UP
> INFO [GossipStage:1] 2012-01-13 00:02:37,353 StorageService.java (line 1016)
> Nodes /50.56.58.55 and action-quick2/50.56.31.186 have the same token
> 85070591730234615865843651857942052864. Ignoring /50.56.58.55
> INFO [GossipTasks:1] 2012-01-13 00:02:47,158 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipStage:1] 2012-01-13 00:02:50,162 Gossiper.java (line 818)
> InetAddress /50.56.58.55 is now dead.
> INFO [GossipStage:1] 2012-01-13 00:02:50,163 StorageService.java (line 1156)
> Removing token 122029383590318827259508597176866581733 for /50.56.58.55
> {noformat}
> in the above, /50.56.58.55 was the replaced IP.
> tried adding the "Gossiper.instance.removeEndpoint(endpoint);" in the
> StorageService.java where the message 'Nodes %s and %s have the same token
> %s. Ignoring %s",' seems only have fixed this temporary. Here is a ring
> output:
> {noformat}
> riptano@action-quick:~/work/cassandra$ ./bin/nodetool -h localhost ring
> Address DC RackStatus State LoadOwns
> Token
>
> 85070591730234615865843651857942052864
> 50.56.59.68 datacenter1 rack1 Up Normal 6.67 KB 85.56%
> 60502102442797279294142560823234402248
> 50.56.31.186datacenter1 rack1 Up Normal 11.12 KB14.44%
> 85070591730234615865843651857942052864
> {noformat}
> gossipinfo:
> {noformat}
> $ ./bin/nodetool -h localhost gossipinfo
> /50.56.58.55
> LOAD:6835.0
> SCHEMA:--1000--
> RPC_AD
[jira] [Created] (CASSANDRA-4742) key cache exist for dropped column families
Tamar Fraenkel created CASSANDRA-4742: - Summary: key cache exist for dropped column families Key: CASSANDRA-4742 URL: https://issues.apache.org/jira/browse/CASSANDRA-4742 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.8 Environment: DataStax AMI on Amazon Reporter: Tamar Fraenkel I have couple of dropped column families, and in the JMX console I don't see them under org.apache.cassandra.db.ColumnFamilies, BUT I do see them under org.apache.cassandra.db.Caches, and the cache is not empty! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
