[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-09-28 Thread Donald Smith (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151127#comment-14151127
 ] 

Donald Smith commented on CASSANDRA-:
-

I know this is moot because of the redesign of hints, but I want to understand 
this. OK, if a hint was successfully delivered, then I can see how a tombstone 
would be useful for causing deletion of *older* instances in other sstables.  
But if a hint timed-out (tombstone), then any older instance will also have 
timed out (presumably). So, could tombstones be deleted in that case (timeout)? 
 Perhaps a timed out cell IS a tombstone, but my point is: I don't see why they 
need to take up space.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-09-19 Thread Donald Smith (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141239#comment-14141239
 ] 

Donald Smith commented on CASSANDRA-:
-

Perhaps this question is inappropriate here. But can tombstones be completely 
omitted for system.hints, given that they're not replicated and given that only 
internal code modifies them in normal operation?  If a hint is delivered 
successfully, why does it need a tombstone at all?  If it times out, then 
cassandra is going to give up on delivering it. So, again, why does it need a 
tombstone?   On the cassandra irc channel, several people speculated that 
the cassandra developers didn't want to make a *special case* for system.hints. 
  Also, system.hints has *gc_grace_seconds=0*, so they won't survive a 
compaction, presumably.  I realize that in C* 3.0, tombstones will be moved out 
of tables, but I still am perplexed why tombstones are needed at all for hints. 
  My apologies if this is a dumb question.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 2.0.11

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-09-19 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141256#comment-14141256
 ] 

Aleksey Yeschenko commented on CASSANDRA-:
--

You can't just omit them, because multiple sstables.

Anyway, I'm closing the ticket as duplicate of CASSANDRA-6230, because there is 
no bullet-proof way to fix the issue until then, unfortunately.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-09-19 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141257#comment-14141257
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

How else would you mark it delivered?

(See CASSANDRA-6230 for a fix that is out of scope here.)

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-07-29 Thread Vishal Mehta (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078171#comment-14078171
 ] 

Vishal Mehta commented on CASSANDRA-:
-

Hello Every,

Please pardon my ignorance, since I am writing first time in opensource bug 
report.

Recently I think I hit this bug because I saw similar symptoms in my 3 node 
cassandra setup. Where I am running a test with around 12K qps (inserts in 3 
different tables) with TTL set to 1 hour and keyspace has GC seconds set to 
14400 (4 hours).

So tests eventually runs to a point where Cassandra sees Tombstones more than 
100K and it crashes with following exception in 
/var/log/cassandra/cassandra.log.

{noformat}
ERROR 13:23:56,747 Scanned over 10 tombstones in system.hints; query 
aborted (see tombstone_fail_threshold)
ERROR 13:23:56,962 Exception in thread Thread[HintedHandoff:1,1,main]
org.apache.cassandra.db.filter.TombstoneOverwhelmingException
at 
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:202)
at 
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
at 
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
at 
org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
at 
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
at 
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at 
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1547)
at 
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1376)
at 
org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:373)
at 
org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:330)
at 
org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:91)
at 
org.apache.cassandra.db.HintedHandOffManager$5.run(HintedHandOffManager.java:547)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
 INFO 13:24:00,987 No gossip backlog; proceeding
{noformat}

*Note:* Is it plausible to keep GC seconds closer to TTLs? Also I could see one 
of the node deleted all the records from disk and freed up the space, where as 
other two nodes never deleted their tombstones.



 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 2.0.10

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-06-10 Thread Walid Darwish (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026537#comment-14026537
 ] 

Walid Darwish commented on CASSANDRA-:
--


Just wondering why compaction is disabled on the hints table by default (and 
only on that table across the system keyspace)? 
is that for performance reasons? Can this be turned on safely?
Also, can this be changed safely to levelled compaction?

compaction= {'enabled': 'false', 'class': 'SizeTieredCompactionStrategy'}

I am using 2.0.7 




 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.17, 2.0.9

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-06-10 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026545#comment-14026545
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

Because the hints manager does compactions manually, only when it knows it will 
be useful (after hint delivery).  The generic strategies would waste a ton of 
effort, especially LCS.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.17, 2.0.9

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-04-23 Thread Walid Darwish (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978190#comment-13978190
 ] 

Walid Darwish commented on CASSANDRA-:
--

Jonathan, I am following up on this on behalf of Anne. Any luck with the log 
files uploaded?


 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-04-23 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978255#comment-13978255
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

[~wdarwish] Nope.  [~iamaleksey] do you have any explanation for this before we 
debug log the hell out of hasIrrelevantData + shouldPurge?

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-04-23 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978389#comment-13978389
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

The sstable is a single partition with cells like this:

{noformat}
{key: 3933deebcbec4fc5b4be8d7cb5b62e5c,
 columns: 
[[2c0bd697-b391-11e3-96a7-ad88e7f21869:7,1395692184,1395692184185,d],
 
[2c0bd698-b391-11e3-96a7-ad88e7f21869:7,1395692184,1395692184185,d],
 
[2c0bd699-b391-11e3-96a7-ad88e7f21869:7,1395692184,1395692184185,d],
 
[2c0bd69a-b391-11e3-96a7-ad88e7f21869:7,1395692184,1395692184185,d],
 
[2c0bd69b-b391-11e3-96a7-ad88e7f21869:7,1395692184,1395692184185,d],
{noformat}

I.e., a lot of tombstones deleted Mar 24 15:16:24 2014.  So I would have 
expected the compaction that ran at 16:40 to have collected them.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-28 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950812#comment-13950812
 ] 

Anne Sullivan commented on CASSANDRA-:
--

Files uploaded to scp server - thanks.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-19 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13940757#comment-13940757
 ] 

Anne Sullivan commented on CASSANDRA-:
--

Attached debug logs (cassandra_system.log.debug.gz)
- two node cluster, RF=2 (node A / node B)
- bring node B down for 30 minutes, back online @ 2014-03-19 10:15:00
- hinted handoff starts @  2014-03-19 10:18:00
- hinted handoff finishes successfully @ 2014-03-19 10:59:04:

INFO [HintedHandoff:1] 2014-03-19 10:59:04,434 HintedHandOffManager.java (line 
378) Finished hinted handoff of 1903040 rows to endpoint /###.###.###.###

- @ 2014-03-19 10:59:16,428 user defined compaction compacts down to one 
sstable (expected result = no sstables / empty set):

INFO [CompactionExecutor:3421] 2014-03-19 10:59:16,428 CompactionTask.java 
(line 275) Compacted 4 sstables to 
[/path/to/db/data/system/hints/system-hints-jb-49,].  172,043,512 bytes to 
6,365,114 (~3% of original) in 11,991ms = 0.506233MB/s.  4 total partitions 
merged to 1.  Partition merge counts were {4:1, }

- @ 2014-03-19 11:08:08 another hinted handoff starts (presumably because the 
hints table isn't empty, and it's checking every 10 minutes) and reports 0 live 
cells:

WARN [HintedHandoff:2] 2014-03-19 11:08:08,639 SliceQueryFilter.java (line 210) 
Read 0 live and 75750 tombstoned cells in system.hints (see 
tombstone_warn_threshold)
 INFO [HintedHandoff:2] 2014-03-19 11:08:08,639 HintedHandOffManager.java (line 
378) Finished hinted handoff of 0 rows to endpoint /###.###.###.###

- user defined compaction again leaves behind a single sstable:

 INFO [CompactionExecutor:3438] 2014-03-19 11:08:08,963 CompactionTask.java 
(line 275) Compacted 1 sstables to 
[/path/to/db/data/system/hints/system-hints-jb-50,].  6,365,114 bytes to 
6,365,114 (~100% of original) in 322ms = 18.851695MB/s.  1 total partitions 
merged to 1.  Partition merge counts were {1:1, }


Partial output from running cqlsh describe keyspace system, to confirm that 
gc_grace_seconds=0, so expecting tombstones to be removed on compaction:

CREATE TABLE hints (
  target_id uuid,
  hint_id timeuuid,
  message_version int,
  mutation blob,
  PRIMARY KEY (target_id, hint_id, message_version)
) WITH COMPACT STORAGE AND
  bloom_filter_fp_chance=0.01 AND
  caching='KEYS_ONLY' AND
  comment='hints awaiting delivery' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=0 AND
  index_interval=128 AND
  read_repair_chance=0.00 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  default_time_to_live=0 AND
  speculative_retry='99.0PERCENTILE' AND
  memtable_flush_period_in_ms=360 AND
  compaction={'enabled': 'false', 'class': 'SizeTieredCompactionStrategy'} AND
  compression={'sstable_compression': 'LZ4Compressor'};





 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-19 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13941048#comment-13941048
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

So, most of it got compacted away, but the rest did not and even survived 
another compaction.

Can you attach the surviving sstable?  If you're worried about exposing 
sensitive data I can arrange a private location.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt, cassandra_system.log.debug.gz






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-13 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13933837#comment-13933837
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

Can you attach a log where this happens with debug enabled on 
o.a.c.db.compaction?

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-03-03 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918081#comment-13918081
 ] 

Anne Sullivan commented on CASSANDRA-:
--

With this fix, we're seeing a case where the user-defined compaction that runs 
post-hint-replay is not removing the sstables as expected.  The hinted handoff 
finishes successfully, but some sstables remain (100% tombstones) and then the 
10 minute check for hints keeps trying to start HH and failing with the 
tombstone exception.  Running a manual compaction clears them up.  It occurs 
100% of the time if we have some larger sstables in the hints CF ... so 4 small 
sstables ~ 10MB and 2 sstables  100MB, the 2 that are large are never 
processed by the user-defined compaction.  


 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-20 Thread graham sanderson (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907926#comment-13907926
 ] 

graham sanderson commented on CASSANDRA-:
-

We have seen an issue where we get into the 100K tombstone state with the 
hints table also; It has happened more than once (worked around in 2.0.5 with 
manual compaction on all nodes of system.hints). 

One quick question... this patch will hopefully prevent us getting into this 
state in many cases, unless you have a huge number of hints for a node that is 
down for a very long time: since there is no auto-compaction, a large number of 
hints may have expired from TTL, thus preventing any further hint delivery.

This is OK if you bring the node back up as a fresh one, otherwise, I guess 
suggesting manual compaction is fine, or we could do a compact().get() on 
seeing a TombstoneOverwhelmingException?

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-07 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894481#comment-13894481
 ] 

Aleksey Yeschenko commented on CASSANDRA-:
--

LGTM

Nit: 1.2 has DatabaseDescriptor.getTombstoneDebugThreshold(), not 
DatabaseDescriptor.getTombstoneWarnThreshold()


 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-06 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893491#comment-13893491
 ] 

Anne Sullivan commented on CASSANDRA-:
--

Will this patch basically force a compaction?  I think it also becomes a 
problem for nodetool repair - if you run a nodetool repair while the hints are 
being replayed, the repair will fail on reading the hint row if  100K 
tombstones present

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-06 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893503#comment-13893503
 ] 

Anne Sullivan commented on CASSANDRA-:
--

Sorry, just re-read the patch, it will fix the nodetool repair problem too.  
Thanks.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-06 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893502#comment-13893502
 ] 

Jonathan Ellis commented on CASSANDRA-:
---

Hints (and other local system tables) are neither replicated nor repaired.

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Commented] (CASSANDRA-6666) Avoid accumulating tombstones after partial hint replay

2014-02-06 Thread Anne Sullivan (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13893506#comment-13893506
 ] 

Anne Sullivan commented on CASSANDRA-:
--

Didn't realize that, thanks for the info.  I'll have to try to reproduce the 
repair issue then as it must have  been something different causing it.  

 Avoid accumulating tombstones after partial hint replay
 ---

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
  Labels: hintedhandoff
 Fix For: 1.2.16, 2.0.6

 Attachments: .txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)