date:20130425


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641589#comment-13641589
 ] 

Ondřej Černoš commented on CASSANDRA-5432:
--

I have exactly the same issue as Arya.

I also had to open non-SSL ports from within the datacenter in order to create 
the cluster.

I was wondering if it could be a networking issue (we use mixed aws-private 
cloud setup), so it is good to see we are not alone with this.

 Repair Freeze/Gossip Invisibility Issues 1.2.4
 --

 Key: CASSANDRA-5432
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5432
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Ubuntu 10.04.1 LTS
 C* 1.2.3
 Sun Java 6 u43
 JNA Enabled
 Not using VNodes
Reporter: Arya Goudarzi
Assignee: Vijay
Priority: Critical

 Read comment 6. This description summarizes the repair issue only, but I 
 believe there is a bigger problem going on with networking as described on 
 that comment. 
 Since I have upgraded our sandbox cluster, I am unable to run repair on any 
 node and I am reaching our gc_grace seconds this weekend. Please help. So 
 far, I have tried the following suggestions:
 - nodetool scrub
 - offline scrub
 - running repair on each CF separately. Didn't matter. All got stuck the same 
 way.
 The repair command just gets stuck and the machine is idling. Only the 
 following logs are printed for repair job:
  INFO [Thread-42214] 2013-04-05 23:30:27,785 StorageService.java (line 2379) 
 Starting repair command #4, repairing 1 ranges for keyspace 
 cardspring_production
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,789 AntiEntropyService.java 
 (line 652) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] new session: will 
 sync /X.X.X.190, /X.X.X.43, /X.X.X.56 on range 
 (1808575600,42535295865117307932921825930779602032] for 
 keyspace_production.[comma separated list of CFs]
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,790 AntiEntropyService.java 
 (line 858) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] requesting merkle 
 trees for BusinessConnectionIndicesEntries (to [/X.X.X.43, /X.X.X.56, 
 /X.X.X.190])
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,086 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.43
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,147 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.56
 Please advise. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5494) Inconsistent view of nodes in CommandDroppedTasks


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641596#comment-13641596
 ] 

Ondřej Černoš commented on CASSANDRA-5494:
--

If only those where drops were detected are reported, why are there the zeros?

 Inconsistent view of nodes in CommandDroppedTasks
 -

 Key: CASSANDRA-5494
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5494
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.3
Reporter: Ondřej Černoš

 Please see related CASSANDRA-5493 for context.
 On our testing platform we saw output from {{get CommandDroppedTasks}} in 
 jmxterm, which was different on different DCs. In DC1, we saw 3x public IPs 
 of remote DC nodes, 2x public IPs of local DC nodes, 2x private IPs of local 
 DC nodes. We don't see stats for the very node the command is run on.
 On the other hand, on the remote DC, we see stats for the node the command is 
 run on, stored under the node's public IP.
 This is at least confusing, maybe even a bug.
 After restarting all the nodes in the cluster (both datacenters) the 
 situation settled - now we don't see stats for the node the command is run 
 on. This is however inconsistent with the situation on production (see 
 CASSANDRA-5493).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5493) Confusing output of CommandDroppedTasks


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641598#comment-13641598
 ] 

Ondřej Černoš commented on CASSANDRA-5493:
--

Please see CASSANDRA-5432, that is the problem we were debugging.

 Confusing output of CommandDroppedTasks
 ---

 Key: CASSANDRA-5493
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5493
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.3
Reporter: Ondřej Černoš

 We have 2 DCs, 3 nodes in each, using EC2 support. We are debugging nodetool 
 repair problems (roughly 1 out of 2 attempts just freezes). We looked into 
 the MessagingServiceBean to see what is going on using jmxterm. See the 
 following:
 {noformat}
 #mbean = org.apache.cassandra.net:type=MessagingService:
 CommandDroppedTasks = { 
  107.aaa.bbb.ccc = 0;
  166.ddd.eee.fff = 124320;
  10.ggg.hhh.iii = 0;
  107.jjj.kkk.lll = 0;
  166.mmm.nnn.ooo = 1336699;
  166.ppp.qqq.rrr = 1329171;
  10.sss.ttt.uuu = 0;
  107.vvv.www.xxx = 0;
 };
 {noformat}
 The problem with this output is it has 8 records. The node's neighbours (the 
 107 and 10 nodes) are mentioned twice in the output, once with their public 
 IPs and once with their private IPs. The nodes in remote DC (the 166 ones) 
 are reported only once. I am pretty sure this is a bug - the node should be 
 reported only with one of its addresses in all outputs from Cassandra and it 
 should be consistent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5432) Repair Freeze/Gossip Invisibility Issues 1.2.4


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641599#comment-13641599
 ] 

Ondřej Černoš commented on CASSANDRA-5432:
--

Please see also CASSANDRA-5493 - the MessagingService also reports dropped 
messages on _itself_ using it's public IP. The output displays 3 public IPs and 
2 private (the private IP of the node itself is not included), while the remote 
DC is reported correctly. This seems related.

 Repair Freeze/Gossip Invisibility Issues 1.2.4
 --

 Key: CASSANDRA-5432
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5432
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Ubuntu 10.04.1 LTS
 C* 1.2.3
 Sun Java 6 u43
 JNA Enabled
 Not using VNodes
Reporter: Arya Goudarzi
Assignee: Vijay
Priority: Critical

 Read comment 6. This description summarizes the repair issue only, but I 
 believe there is a bigger problem going on with networking as described on 
 that comment. 
 Since I have upgraded our sandbox cluster, I am unable to run repair on any 
 node and I am reaching our gc_grace seconds this weekend. Please help. So 
 far, I have tried the following suggestions:
 - nodetool scrub
 - offline scrub
 - running repair on each CF separately. Didn't matter. All got stuck the same 
 way.
 The repair command just gets stuck and the machine is idling. Only the 
 following logs are printed for repair job:
  INFO [Thread-42214] 2013-04-05 23:30:27,785 StorageService.java (line 2379) 
 Starting repair command #4, repairing 1 ranges for keyspace 
 cardspring_production
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,789 AntiEntropyService.java 
 (line 652) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] new session: will 
 sync /X.X.X.190, /X.X.X.43, /X.X.X.56 on range 
 (1808575600,42535295865117307932921825930779602032] for 
 keyspace_production.[comma separated list of CFs]
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,790 AntiEntropyService.java 
 (line 858) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] requesting merkle 
 trees for BusinessConnectionIndicesEntries (to [/X.X.X.43, /X.X.X.56, 
 /X.X.X.190])
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,086 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.43
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,147 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.56
 Please advise. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5443) Add CAS CQL support

2013-04-25 Thread Sylvain Lebresne (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641701#comment-13641701
 ] 

Sylvain Lebresne commented on CASSANDRA-5443:
-

bq. With regard to the result, it would be consistent with SQL to return a 
count of affected logical rows; not a boolean.

I suppose that would work, but it feels to me that returning whether the 
condition did apply is more direct that the number of affected rows.

I understand that the motivation is to mimick SQL more, but then we would need 
to return the number of affected rows for normal updates too (otherwise that 
would feel inconsistent to me). Also, my understanding (that is possibly 
broken) is that in SQL the number of affected rows is returned as some metadata 
of the query (typically, in JDBC, as the result value of executeUpdate()), not 
as a result set as we would do here, so would we be really fully consistent 
with SQL anyway? 

Anyway, not that I'm really against the idea, just wondering.


 Add CAS CQL support
 ---

 Key: CASSANDRA-5443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5443
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5467) isRunning flag set prematurely in org.apache.cassandra.transport.Server

2013-04-25 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641736#comment-13641736
 ] 

Marcus Eriksson commented on CASSANDRA-5467:


lgtm

 isRunning flag set prematurely in org.apache.cassandra.transport.Server
 ---

 Key: CASSANDRA-5467
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5467
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.2
Reporter: John Sanda
Assignee: Sylvain Lebresne
Priority: Minor
  Labels: jmx, server
 Fix For: 1.2.5

 Attachments: 5467.txt


 In org.apache.cassandra.transport.Server, the start() method sets the 
 isRunning flag before calling the run() method. In the event of an 
 initialization error like a port conflict an exception will be thrown at line 
 136 which is,
 Channel channel = bootstrap.bind(socket);
 It seems like it might make more sense to set the isRunning flag after 
 binding to the socket. I have a tool that deploys a node and then verifies it 
 is ready to receive CQL requests. I do this via JMX. Unless I use a delay 
 before making that check, the JMX call will return true even though there is 
 a port conflict. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5494) Inconsistent view of nodes in CommandDroppedTasks


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641812#comment-13641812
 ] 

Jonathan Ellis commented on CASSANDRA-5494:
---

https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/MessagingService.java

 Inconsistent view of nodes in CommandDroppedTasks
 -

 Key: CASSANDRA-5494
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5494
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.3
Reporter: Ondřej Černoš

 Please see related CASSANDRA-5493 for context.
 On our testing platform we saw output from {{get CommandDroppedTasks}} in 
 jmxterm, which was different on different DCs. In DC1, we saw 3x public IPs 
 of remote DC nodes, 2x public IPs of local DC nodes, 2x private IPs of local 
 DC nodes. We don't see stats for the very node the command is run on.
 On the other hand, on the remote DC, we see stats for the node the command is 
 run on, stored under the node's public IP.
 This is at least confusing, maybe even a bug.
 After restarting all the nodes in the cluster (both datacenters) the 
 situation settled - now we don't see stats for the node the command is run 
 on. This is however inconsistent with the situation on production (see 
 CASSANDRA-5493).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641881#comment-13641881
 ] 

Jonathan Ellis commented on CASSANDRA-5511:
---

Updates pushed to https://github.com/jbellis/cassandra/commits/5511-2.  Done 
except for LLMT, I think.

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0

[
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641890#comment-13641890
]

Jonathan Ellis commented on CASSANDRA-5511:
---

Oops, missed Marcus's update somehow. Will have a look.

Clean up backwards compatibility complexity for 2.0
---

Key: CASSANDRA-5511
URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Fix For: 2.0

Attachments: CASSANDRA-5511-on_1.2.patch,
CASSANDRA-5511-on_trunk.patch

We've supported rolling upgrades (network-compatible for read/write
operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required
being on a recent release of the immediately prior major series for this to
work as desired.
Meanwhile, we still support reading sstables at least back to 0.6 and
possibly even earlier. This makes dealing with changes to the sstable quite
challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
2.0 is a good place to drop support for sstables older than 1.2.5. Our
experience with network compatibility demonstrates that this is not an
unreasonable burden to impose, and the major version number change suggests
that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5426) Redesign repair messages

2013-04-25 Thread Jeremiah Jordan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641899#comment-13641899
 ] 

Jeremiah Jordan commented on CASSANDRA-5426:


Could any of this be added to other processes using streaming?  
Bootstrap/Decommission/Move/sstableloader?

 Redesign repair messages
 

 Key: CASSANDRA-5426
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5426
 Project: Cassandra
  Issue Type: Improvement
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
  Labels: repair
 Fix For: 2.0


 Many people have been reporting 'repair hang' when something goes wrong.
 Two major causes of hang are 1) validation failure and 2) streaming failure.
 Currently, when those failures happen, the failed node would not respond back 
 to the repair initiator.
 The goal of this ticket is to redesign message flows around repair so that 
 repair never hang.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5514) Allow timestamp hints

Jonathan Ellis created CASSANDRA-5514:
-

 Summary: Allow timestamp hints
 Key: CASSANDRA-5514
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
 Fix For: 2.0


Slice queries can't optimize based on timestamp except for rare cases 
(CASSANDRA-4116).  However, many common queries involve an implicit time 
component, where the application author knows that he is only interested in 
data more recent than X, or older than Y.

We could use the per-sstable max and min timestamps we track to avoid touching 
cold data if we could pass a hint to Cassandra about the time range we care 
about.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5514) Allow timestamp hints

2013-04-25 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641905#comment-13641905
 ] 

Marcus Eriksson commented on CASSANDRA-5514:


i like it, it would go very nice with CASSANDRA-5228 and an age tiered 
compaction strategy

 Allow timestamp hints
 -

 Key: CASSANDRA-5514
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
 Fix For: 2.0


 Slice queries can't optimize based on timestamp except for rare cases 
 (CASSANDRA-4116).  However, many common queries involve an implicit time 
 component, where the application author knows that he is only interested in 
 data more recent than X, or older than Y.
 We could use the per-sstable max and min timestamps we track to avoid 
 touching cold data if we could pass a hint to Cassandra about the time range 
 we care about.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5514) Allow timestamp hints

[
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641906#comment-13641906
]

Jonathan Ellis commented on CASSANDRA-5514:
---

Syntax could be something like

{code}
SELECT *
FROM mytimeseries
WHERE key = foo
USING TIMESTAMP X
{code}

Alternatives could include

# Adding this to the WHERE clause. However, this would be incompatible with
any existing columns named timestamp, and implementation would likely be
messier than just letting the parser pull this out for us. Additionally, we've
already decided to not pretend timestamp is a pseudo-column for insert/update.
# Using the {{WITH}} keyword or a new keyword entirely. This may be somewhere
more natural than {{USING}}, but again, having gone with {{USING}} on
insert/update, it seems like a good idea to be consistent here.

Allow timestamp hints
-

Key: CASSANDRA-5514
URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Fix For: 2.0

Slice queries can't optimize based on timestamp except for rare cases
(CASSANDRA-4116). However, many common queries involve an implicit time
component, where the application author knows that he is only interested in
data more recent than X, or older than Y.
We could use the per-sstable max and min timestamps we track to avoid
touching cold data if we could pass a hint to Cassandra about the time range
we care about.

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641907#comment-13641907
 ] 

Jonathan Ellis commented on CASSANDRA-5511:
---

Marcus's patches LGTM.

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-5514) Allow timestamp hints


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reassigned CASSANDRA-5514:
-

Assignee: Marcus Eriksson

 Allow timestamp hints
 -

 Key: CASSANDRA-5514
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
 Fix For: 2.0


 Slice queries can't optimize based on timestamp except for rare cases 
 (CASSANDRA-4116).  However, many common queries involve an implicit time 
 component, where the application author knows that he is only interested in 
 data more recent than X, or older than Y.
 We could use the per-sstable max and min timestamps we track to avoid 
 touching cold data if we could pass a hint to Cassandra about the time range 
 we care about.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Add helper to create sstables for legacy leveled manifest test

Updated Branches:
  refs/heads/cassandra-1.2 3c93e8c6b - 591b8ca4b


Add helper to create sstables for legacy leveled manifest test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/591b8ca4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/591b8ca4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/591b8ca4

Branch: refs/heads/cassandra-1.2
Commit: 591b8ca4b3cd1074f6c62414975950a69149748d
Parents: 3c93e8c6
Author: Marcus Eriksson marc...@spotify.com
Authored: Thu Apr 25 18:04:12 2013 +0200
Committer: Marcus Eriksson marc...@spotify.com
Committed: Thu Apr 25 18:04:12 2013 +0200

--
 test/unit/org/apache/cassandra/SchemaLoader.java   |3 +
 .../LegacyLeveledManifestTestHelper.java   |   57 +++
 2 files changed, 60 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/591b8ca4/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 3db1fc5..b2ae83a 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -220,6 +220,9 @@ public class SchemaLoader
   null),
standardCFMD(ks1, 
StandardLeveled, withOldCfIds)

.compactionStrategyClass(LeveledCompactionStrategy.class)
+   
.compactionStrategyOptions(leveledOptions),
+   standardCFMD(ks1, legacyleveled, 
withOldCfIds)
+   
.compactionStrategyClass(LeveledCompactionStrategy.class)

.compactionStrategyOptions(leveledOptions)));
 
 // Keyspace 2

http://git-wip-us.apache.org/repos/asf/cassandra/blob/591b8ca4/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
 
b/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
new file mode 100644
index 000..9ce60ea
--- /dev/null
+++ 
b/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
@@ -0,0 +1,57 @@
+package org.apache.cassandra.db.compaction;
+
+
+import java.io.File;
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.Set;
+import org.junit.Ignore;
+import org.junit.Test;
+
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.SSTableReader;
+import org.apache.cassandra.io.sstable.SSTableUtils;
+import org.apache.cassandra.io.util.FileUtils;
+
+@Ignore
+public class LegacyLeveledManifestTestHelper extends SchemaLoader
+{
+public final static String PROP = migration-sstable-root;
+public final static String KS = Keyspace1;
+public final static String CF = legacyleveled;
+/**
+ * Generates two sstables to be used to test migrating from a .json 
manifest to keeping the level in the sstable
+ * metadata.
+ *
+ * Do this:
+ * 1. remove @Ignore
+ * 2. comment out the @Before and @After methods above
+ * 3. run this method
+ * 4. checkout trunk
+ * 5. copy the .json file from the previous version to the current one
+ *(ie; 
test/data/migration-sstables/ic/Keyspace1/legacyleveled/legacyleveled.json)
+ * 6. update LegacyLeveledManifestTest to use the new version.
+ */
+@Test
+public void generateSSTable() throws IOException
+{
+File legacySSTableDir = 
getLegacySSTableDir(Descriptor.Version.current_version);
+FileUtils.createDirectory(legacySSTableDir);
+SetString keys = new HashSetString();
+for(int i = 0; i  10; i++)
+{
+keys.add(key+i);
+}
+for(int i = 0; i  3; i++)
+{
+SSTableReader ssTable = 
SSTableUtils.prepare().ks(KS).cf(CF).dest(new Descriptor(legacySSTableDir, KS, 
CF, i, false)).write(keys);
+System.out.println(ssTable);
+}
+}
+public static File getLegacySSTableDir(String version)
+{
+return new File(System.getProperty(PROP) + File.separator + version + 
File.separator + KS + File.separator + CF);
+}
+
+}

[1/3] git commit: Add helper to create sstables for legacy leveled manifest test

Updated Branches:
  refs/heads/trunk bf2ee0443 - 32f4e2cce


Add helper to create sstables for legacy leveled manifest test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/591b8ca4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/591b8ca4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/591b8ca4

Branch: refs/heads/trunk
Commit: 591b8ca4b3cd1074f6c62414975950a69149748d
Parents: 3c93e8c6
Author: Marcus Eriksson marc...@spotify.com
Authored: Thu Apr 25 18:04:12 2013 +0200
Committer: Marcus Eriksson marc...@spotify.com
Committed: Thu Apr 25 18:04:12 2013 +0200

--
 test/unit/org/apache/cassandra/SchemaLoader.java   |3 +
 .../LegacyLeveledManifestTestHelper.java   |   57 +++
 2 files changed, 60 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/591b8ca4/test/unit/org/apache/cassandra/SchemaLoader.java
--
diff --git a/test/unit/org/apache/cassandra/SchemaLoader.java 
b/test/unit/org/apache/cassandra/SchemaLoader.java
index 3db1fc5..b2ae83a 100644
--- a/test/unit/org/apache/cassandra/SchemaLoader.java
+++ b/test/unit/org/apache/cassandra/SchemaLoader.java
@@ -220,6 +220,9 @@ public class SchemaLoader
   null),
standardCFMD(ks1, 
StandardLeveled, withOldCfIds)

.compactionStrategyClass(LeveledCompactionStrategy.class)
+   
.compactionStrategyOptions(leveledOptions),
+   standardCFMD(ks1, legacyleveled, 
withOldCfIds)
+   
.compactionStrategyClass(LeveledCompactionStrategy.class)

.compactionStrategyOptions(leveledOptions)));
 
 // Keyspace 2

http://git-wip-us.apache.org/repos/asf/cassandra/blob/591b8ca4/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
 
b/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
new file mode 100644
index 000..9ce60ea
--- /dev/null
+++ 
b/test/unit/org/apache/cassandra/db/compaction/LegacyLeveledManifestTestHelper.java
@@ -0,0 +1,57 @@
+package org.apache.cassandra.db.compaction;
+
+
+import java.io.File;
+import java.io.IOException;
+import java.util.HashSet;
+import java.util.Set;
+import org.junit.Ignore;
+import org.junit.Test;
+
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.SSTableReader;
+import org.apache.cassandra.io.sstable.SSTableUtils;
+import org.apache.cassandra.io.util.FileUtils;
+
+@Ignore
+public class LegacyLeveledManifestTestHelper extends SchemaLoader
+{
+public final static String PROP = migration-sstable-root;
+public final static String KS = Keyspace1;
+public final static String CF = legacyleveled;
+/**
+ * Generates two sstables to be used to test migrating from a .json 
manifest to keeping the level in the sstable
+ * metadata.
+ *
+ * Do this:
+ * 1. remove @Ignore
+ * 2. comment out the @Before and @After methods above
+ * 3. run this method
+ * 4. checkout trunk
+ * 5. copy the .json file from the previous version to the current one
+ *(ie; 
test/data/migration-sstables/ic/Keyspace1/legacyleveled/legacyleveled.json)
+ * 6. update LegacyLeveledManifestTest to use the new version.
+ */
+@Test
+public void generateSSTable() throws IOException
+{
+File legacySSTableDir = 
getLegacySSTableDir(Descriptor.Version.current_version);
+FileUtils.createDirectory(legacySSTableDir);
+SetString keys = new HashSetString();
+for(int i = 0; i  10; i++)
+{
+keys.add(key+i);
+}
+for(int i = 0; i  3; i++)
+{
+SSTableReader ssTable = 
SSTableUtils.prepare().ks(KS).cf(CF).dest(new Descriptor(legacySSTableDir, KS, 
CF, i, false)).write(keys);
+System.out.println(ssTable);
+}
+}
+public static File getLegacySSTableDir(String version)
+{
+return new File(System.getProperty(PROP) + File.separator + version + 
File.separator + KS + File.separator + CF);
+}
+
+}

[2/3] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/85b0d150
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/85b0d150
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/85b0d150

Branch: refs/heads/trunk
Commit: 85b0d150fb5cf7892eafd6f090cc176e3634a0af
Parents: bf2ee04 591b8ca
Author: Marcus Eriksson marc...@spotify.com
Authored: Thu Apr 25 18:16:39 2013 +0200
Committer: Marcus Eriksson marc...@spotify.com
Committed: Thu Apr 25 18:16:39 2013 +0200

--
 test/unit/org/apache/cassandra/SchemaLoader.java   |3 +
 .../LegacyLeveledManifestTestHelper.java   |   57 +++
 2 files changed, 60 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/85b0d150/test/unit/org/apache/cassandra/SchemaLoader.java
--

[3/3] git commit: Use new ic legacy sstables for legacy leveled manifest test

Use new ic legacy sstables for legacy leveled manifest test


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32f4e2cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32f4e2cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32f4e2cc

Branch: refs/heads/trunk
Commit: 32f4e2ccec259c3a7c9527b538d53464c52cfc11
Parents: 85b0d15
Author: Marcus Eriksson marc...@spotify.com
Authored: Thu Apr 25 18:17:53 2013 +0200
Committer: Marcus Eriksson marc...@spotify.com
Committed: Thu Apr 25 18:20:12 2013 +0200

--
 ...Keyspace1-legacyleveled-hf-0-CompressionInfo.db |  Bin 46 - 0 bytes
 .../Keyspace1/Keyspace1-legacyleveled-hf-0-Data.db |  Bin 70 - 0 bytes
 .../Keyspace1-legacyleveled-hf-0-Filter.db |  Bin 16 - 0 bytes
 .../Keyspace1-legacyleveled-hf-0-Index.db  |  Bin 14 - 0 bytes
 .../Keyspace1-legacyleveled-hf-0-Statistics.db |  Bin 4340 - 0 bytes
 .../Keyspace1/Keyspace1-legacyleveled-hf-0-TOC.txt |6 ---
 ...Keyspace1-legacyleveled-hf-1-CompressionInfo.db |  Bin 46 - 0 bytes
 .../Keyspace1/Keyspace1-legacyleveled-hf-1-Data.db |  Bin 70 - 0 bytes
 .../Keyspace1-legacyleveled-hf-1-Filter.db |  Bin 16 - 0 bytes
 .../Keyspace1-legacyleveled-hf-1-Index.db  |  Bin 14 - 0 bytes
 .../Keyspace1-legacyleveled-hf-1-Statistics.db |  Bin 4340 - 0 bytes
 ...Keyspace1-legacyleveled-hf-2-CompressionInfo.db |  Bin 46 - 0 bytes
 .../Keyspace1/Keyspace1-legacyleveled-hf-2-Data.db |  Bin 70 - 0 bytes
 .../Keyspace1-legacyleveled-hf-2-Filter.db |  Bin 16 - 0 bytes
 .../Keyspace1-legacyleveled-hf-2-Index.db  |  Bin 14 - 0 bytes
 .../Keyspace1-legacyleveled-hf-2-Statistics.db |  Bin 4340 - 0 bytes
 .../hf/Keyspace1/legacyleveled.json|   27 ---
 .../Keyspace1-legacyleveled-ic-0-Data.db   |  Bin 0 - 530 bytes
 .../Keyspace1-legacyleveled-ic-0-Digest.sha1   |1 +
 .../Keyspace1-legacyleveled-ic-0-Filter.db |  Bin 0 - 24 bytes
 .../Keyspace1-legacyleveled-ic-0-Index.db  |  Bin 0 - 180 bytes
 .../Keyspace1-legacyleveled-ic-0-Statistics.db |  Bin 0 - 4361 bytes
 .../Keyspace1-legacyleveled-ic-0-Summary.db|  Bin 0 - 92 bytes
 .../Keyspace1-legacyleveled-ic-0-TOC.txt   |7 
 .../Keyspace1-legacyleveled-ic-1-Data.db   |  Bin 0 - 530 bytes
 .../Keyspace1-legacyleveled-ic-1-Digest.sha1   |1 +
 .../Keyspace1-legacyleveled-ic-1-Filter.db |  Bin 0 - 24 bytes
 .../Keyspace1-legacyleveled-ic-1-Index.db  |  Bin 0 - 180 bytes
 .../Keyspace1-legacyleveled-ic-1-Statistics.db |  Bin 0 - 4361 bytes
 .../Keyspace1-legacyleveled-ic-1-Summary.db|  Bin 0 - 92 bytes
 .../Keyspace1-legacyleveled-ic-1-TOC.txt   |7 
 .../Keyspace1-legacyleveled-ic-2-Data.db   |  Bin 0 - 530 bytes
 .../Keyspace1-legacyleveled-ic-2-Digest.sha1   |1 +
 .../Keyspace1-legacyleveled-ic-2-Filter.db |  Bin 0 - 24 bytes
 .../Keyspace1-legacyleveled-ic-2-Index.db  |  Bin 0 - 180 bytes
 .../Keyspace1-legacyleveled-ic-2-Statistics.db |  Bin 0 - 4361 bytes
 .../Keyspace1-legacyleveled-ic-2-Summary.db|  Bin 0 - 92 bytes
 .../Keyspace1-legacyleveled-ic-2-TOC.txt   |7 
 .../ic/Keyspace1/legacyleveled/legacyleveled.json  |   27 +++
 .../db/compaction/LegacyLeveledManifestTest.java   |   19 +-
 40 files changed, 61 insertions(+), 42 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f4e2cc/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-CompressionInfo.db
--
diff --git 
a/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-CompressionInfo.db
 
b/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-CompressionInfo.db
deleted file mode 100644
index af783d6..000
Binary files 
a/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-CompressionInfo.db
 and /dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f4e2cc/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-Data.db
--
diff --git 
a/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-Data.db
 
b/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-Data.db
deleted file mode 100644
index 854a1c9..000
Binary files 
a/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-Data.db
 and /dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/32f4e2cc/test/data/migration-sstables/hf/Keyspace1/Keyspace1-legacyleveled-hf-0-Filter.db

[jira] [Created] (CASSANDRA-5515) Track sstable coldness

Jonathan Ellis created CASSANDRA-5515:
-

 Summary: Track sstable coldness
 Key: CASSANDRA-5515
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5515
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
 Fix For: 2.0


Keeping a count of reads per-sstable would allow STCS to automatically ignore 
cold data rather than recompacting it constantly with hot data, dramatically 
reducing compaction load for typical time series applications and others with 
time-correlated access patterns.  We would not need a separate age-tiered 
compaction strategy.

(This will really be useful in conjunction with CASSANDRA-5514.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5514) Allow timestamp hints


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641939#comment-13641939
 ] 

Jonathan Ellis commented on CASSANDRA-5514:
---

Sylvain suggests that instead of adding extra syntax, we track min and max 
column values (cell name components) on a per-sstable basis.  This requires 
less moving pieces to implement (don't need to touch QueryProcessor) and I'm 
generally a fan of less special cases.

A small downside is that we'll need to make SSTableMetadata.Collector 
cql-aware.  Simplest approach IMO is to just track all the clustering 
components per-sstable, it's not going to be a whole lot of extra data consumed.

 Allow timestamp hints
 -

 Key: CASSANDRA-5514
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
 Fix For: 2.0


 Slice queries can't optimize based on timestamp except for rare cases 
 (CASSANDRA-4116).  However, many common queries involve an implicit time 
 component, where the application author knows that he is only interested in 
 data more recent than X, or older than Y.
 We could use the per-sstable max and min timestamps we track to avoid 
 touching cold data if we could pass a hint to Cassandra about the time range 
 we care about.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5514) Allow timestamp hints

[
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641939#comment-13641939
]

Jonathan Ellis edited comment on CASSANDRA-5514 at 4/25/13 4:46 PM:

Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. The query then
just becomes SELECT ... WHERE mytimecolumn X. This requires less moving
pieces to implement (don't need to touch QueryProcessor) and I'm generally a
fan of less special cases.

A small downside is that we'll need to make SSTableMetadata.Collector
cql-aware. Simplest approach IMO is to just track all the clustering
components per-sstable, it's not going to be a whole lot of extra data consumed.

was (Author: jbellis):
Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. This requires
less moving pieces to implement (don't need to touch QueryProcessor) and I'm
generally a fan of less special cases.

Allow timestamp hints
-

Key: CASSANDRA-5514
URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
Fix For: 2.0

[jira] [Comment Edited] (CASSANDRA-5514) Allow timestamp hints

[
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641939#comment-13641939
]

Jonathan Ellis edited comment on CASSANDRA-5514 at 4/25/13 4:46 PM:

Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. The query then
just becomes

{code}
SELECT *
FROM mytimeseries
WHERE key = foo
AND mytimecolumn X
{code}

This requires less moving pieces to implement (don't need to touch
QueryProcessor) and I'm generally a fan of less special cases.

was (Author: jbellis):
Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. The query then
just becomes SELECT ... WHERE mytimecolumn X. This requires less moving
pieces to implement (don't need to touch QueryProcessor) and I'm generally a
fan of less special cases.

Allow timestamp hints
-

[jira] [Comment Edited] (CASSANDRA-5514) Allow timestamp hints

[
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641939#comment-13641939
]

Jonathan Ellis edited comment on CASSANDRA-5514 at 4/25/13 4:48 PM:

Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. The query then
just becomes

{code}
SELECT *
FROM mytimeseries
WHERE key = foo
AND mytimecolumn X
{code}

This requires less moving pieces to implement (don't need to touch
QueryProcessor) and I'm generally a fan of less special cases.

This also generalizes to non-time-based clustering. I can't think of any
examples off the top of my head where that's critical, but it does seem like a
good option to support. :)

was (Author: jbellis):
Sylvain suggests that instead of adding extra syntax, we track min and max
column values (cell name components) on a per-sstable basis. The query then
just becomes

{code}
SELECT *
FROM mytimeseries
WHERE key = foo
AND mytimecolumn X
{code}

This requires less moving pieces to implement (don't need to touch
QueryProcessor) and I'm generally a fan of less special cases.

Allow timestamp hints
-

[1/3] git commit: update buildTestSSTable to compile

Updated Branches:
  refs/heads/cassandra-1.2 591b8ca4b - 49ec3e804
  refs/heads/trunk 32f4e2cce - 2fa332176


update buildTestSSTable to compile


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49ec3e80
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49ec3e80
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49ec3e80

Branch: refs/heads/cassandra-1.2
Commit: 49ec3e804d1db54fdba83437a9f837e0120661b6
Parents: 591b8ca
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue Apr 23 23:26:54 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Apr 25 11:54:48 2013 -0500

--
 .../cassandra/io/sstable/LegacySSTableTest.java|2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49ec3e80/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java 
b/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
index a31ebb6..db3f1c5 100644
--- a/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
@@ -75,7 +75,7 @@ public class LegacySSTableTest extends SchemaLoader
 public void buildTestSSTable() throws IOException
 {
 // write the output in a version specific directory
-Descriptor dest = getDescriptor(Descriptor.CURRENT_VERSION);
+Descriptor dest = getDescriptor(Descriptor.Version.current_version);
 assert dest.directory.mkdirs() : Could not create  + dest.directory 
+ . Might it already exist?;
 
 SSTableReader ssTable = 
SSTableUtils.prepare().ks(KSNAME).cf(CFNAME).dest(dest).write(TEST_DATA);

[2/3] git commit: update buildTestSSTable to compile

update buildTestSSTable to compile


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49ec3e80
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49ec3e80
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49ec3e80

Branch: refs/heads/trunk
Commit: 49ec3e804d1db54fdba83437a9f837e0120661b6
Parents: 591b8ca
Author: Jonathan Ellis jbel...@apache.org
Authored: Tue Apr 23 23:26:54 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Apr 25 11:54:48 2013 -0500

--
 .../cassandra/io/sstable/LegacySSTableTest.java|2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49ec3e80/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java 
b/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
index a31ebb6..db3f1c5 100644
--- a/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/LegacySSTableTest.java
@@ -75,7 +75,7 @@ public class LegacySSTableTest extends SchemaLoader
 public void buildTestSSTable() throws IOException
 {
 // write the output in a version specific directory
-Descriptor dest = getDescriptor(Descriptor.CURRENT_VERSION);
+Descriptor dest = getDescriptor(Descriptor.Version.current_version);
 assert dest.directory.mkdirs() : Could not create  + dest.directory 
+ . Might it already exist?;
 
 SSTableReader ssTable = 
SSTableUtils.prepare().ks(KSNAME).cf(CFNAME).dest(dest).write(TEST_DATA);

[1/3] git commit: Fix repair -snapshot not working patch by yukim; reviewed by jbellis for CASSANDRA-5512

2013-04-25 Thread yukim

Updated Branches:
  refs/heads/cassandra-1.2 49ec3e804 - 68444f9b2
  refs/heads/trunk 2fa332176 - 5d821a1ea


Fix repair -snapshot not working patch by yukim; reviewed by jbellis for 
CASSANDRA-5512


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68444f9b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68444f9b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68444f9b

Branch: refs/heads/cassandra-1.2
Commit: 68444f9b2f573425bdd38b378e14cd6a8c16821c
Parents: 49ec3e8
Author: Yuki Morishita yu...@apache.org
Authored: Thu Apr 25 12:35:18 2013 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Thu Apr 25 12:35:18 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/net/MessagingService.java |1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/68444f9b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d43fe6..d1b59bb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -10,6 +10,7 @@
  * Don't provide oldCfId for post-1.1 system cfs (CASSANDRA-5490)
  * Fix primary range ignores replication strategy (CASSANDRA-5424)
  * Fix shutdown of binary protocol server (CASSANDRA-5507)
+ * Fix repair -snapshot not working (CASSANDRA-5512)
 Merged from 1.1
  * Add retry mechanism to OTC for non-droppable_verbs (CASSANDRA-5393)
  * Use allocator information to improve memtable memory usage estimate 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/68444f9b/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index cdc8b83..824ac90 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -191,6 +191,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 put(Verb.INDEX_SCAN, IndexScanCommand.serializer);
 put(Verb.REPLICATION_FINISHED, null);
 put(Verb.COUNTER_MUTATION, CounterMutation.serializer);
+put(Verb.SNAPSHOT, SnapshotCommand.serializer);
 }};
 
 /**

[2/3] git commit: Fix repair -snapshot not working patch by yukim; reviewed by jbellis for CASSANDRA-5512

2013-04-25 Thread yukim

Fix repair -snapshot not working patch by yukim; reviewed by jbellis for 
CASSANDRA-5512


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68444f9b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68444f9b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68444f9b

Branch: refs/heads/trunk
Commit: 68444f9b2f573425bdd38b378e14cd6a8c16821c
Parents: 49ec3e8
Author: Yuki Morishita yu...@apache.org
Authored: Thu Apr 25 12:35:18 2013 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Thu Apr 25 12:35:18 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/net/MessagingService.java |1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/68444f9b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d43fe6..d1b59bb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -10,6 +10,7 @@
  * Don't provide oldCfId for post-1.1 system cfs (CASSANDRA-5490)
  * Fix primary range ignores replication strategy (CASSANDRA-5424)
  * Fix shutdown of binary protocol server (CASSANDRA-5507)
+ * Fix repair -snapshot not working (CASSANDRA-5512)
 Merged from 1.1
  * Add retry mechanism to OTC for non-droppable_verbs (CASSANDRA-5393)
  * Use allocator information to improve memtable memory usage estimate 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/68444f9b/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index cdc8b83..824ac90 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -191,6 +191,7 @@ public final class MessagingService implements 
MessagingServiceMBean
 put(Verb.INDEX_SCAN, IndexScanCommand.serializer);
 put(Verb.REPLICATION_FINISHED, null);
 put(Verb.COUNTER_MUTATION, CounterMutation.serializer);
+put(Verb.SNAPSHOT, SnapshotCommand.serializer);
 }};
 
 /**

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

2013-04-25 Thread yukim

Merge branch 'cassandra-1.2' into trunk

Conflicts:
src/java/org/apache/cassandra/net/MessagingService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d821a1e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d821a1e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d821a1e

Branch: refs/heads/trunk
Commit: 5d821a1ea4f87679a8dfc4c8efd2c649990e1594
Parents: 2fa3321 68444f9
Author: Yuki Morishita yu...@apache.org
Authored: Thu Apr 25 12:38:27 2013 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Thu Apr 25 12:38:27 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/net/MessagingService.java |1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d821a1e/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5d821a1e/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --cc src/java/org/apache/cassandra/net/MessagingService.java
index c23f566,824ac90..32b0705
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@@ -212,10 -191,7 +212,11 @@@ public final class MessagingService imp
  put(Verb.INDEX_SCAN, IndexScanCommand.serializer);
  put(Verb.REPLICATION_FINISHED, null);
  put(Verb.COUNTER_MUTATION, CounterMutation.serializer);
+ put(Verb.SNAPSHOT, SnapshotCommand.serializer);
 +put(Verb.ECHO, EchoMessage.serializer);
 +put(Verb.PAXOS_PREPARE, Commit.serializer);
 +put(Verb.PAXOS_PROPOSE, Commit.serializer);
 +put(Verb.PAXOS_COMMIT, Commit.serializer);
  }};
  
  /**

[jira] [Commented] (CASSANDRA-5426) Redesign repair messages

2013-04-25 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642116#comment-13642116
 ] 

Yuki Morishita commented on CASSANDRA-5426:
---

[~jjordan] For the streaming improvements, I'm working on CASSANDRA-5286.

 Redesign repair messages
 

 Key: CASSANDRA-5426
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5426
 Project: Cassandra
  Issue Type: Improvement
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
  Labels: repair
 Fix For: 2.0


 Many people have been reporting 'repair hang' when something goes wrong.
 Two major causes of hang are 1) validation failure and 2) streaming failure.
 Currently, when those failures happen, the failed node would not respond back 
 to the repair initiator.
 The goal of this ticket is to redesign message flows around repair so that 
 repair never hang.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642128#comment-13642128
 ] 

Jonathan Ellis commented on CASSANDRA-5511:
---

Rebased and pushed to https://github.com/jbellis/cassandra/commits/5511-3.  
Tests pass.

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5511:
--

Reviewer: krummas

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (CASSANDRA-5492) Backport row-level bloom filter removal to 1.2


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis reopened CASSANDRA-5492:
---


 Backport row-level bloom filter removal to 1.2
 --

 Key: CASSANDRA-5492
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5492
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.2.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.2.5


 With the possible presence of range tombstones, it is erroneous to skip 
 checking for a given column in SSTableNamesIterator because the bloom filter 
 says it is not there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5492) Backport row-level bloom filter removal to 1.2


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5492:
--

Tester: enigmacurry

 Backport row-level bloom filter removal to 1.2
 --

 Key: CASSANDRA-5492
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5492
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.2.0
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.2.5


 With the possible presence of range tombstones, it is erroneous to skip 
 checking for a given column in SSTableNamesIterator because the bloom filter 
 says it is not there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5432) Repair Freeze/Gossip Invisibility Issues 1.2.4

2013-04-25 Thread Vijay (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-5432:
-

Attachment: 0001-CASSANDRA-5432.patch

attached reverts CASSANDRA-5171 and adds handling of local endpoint.

Arya, mind testing this?

 Repair Freeze/Gossip Invisibility Issues 1.2.4
 --

 Key: CASSANDRA-5432
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5432
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Ubuntu 10.04.1 LTS
 C* 1.2.3
 Sun Java 6 u43
 JNA Enabled
 Not using VNodes
Reporter: Arya Goudarzi
Assignee: Vijay
Priority: Critical
 Attachments: 0001-CASSANDRA-5432.patch


 Read comment 6. This description summarizes the repair issue only, but I 
 believe there is a bigger problem going on with networking as described on 
 that comment. 
 Since I have upgraded our sandbox cluster, I am unable to run repair on any 
 node and I am reaching our gc_grace seconds this weekend. Please help. So 
 far, I have tried the following suggestions:
 - nodetool scrub
 - offline scrub
 - running repair on each CF separately. Didn't matter. All got stuck the same 
 way.
 The repair command just gets stuck and the machine is idling. Only the 
 following logs are printed for repair job:
  INFO [Thread-42214] 2013-04-05 23:30:27,785 StorageService.java (line 2379) 
 Starting repair command #4, repairing 1 ranges for keyspace 
 cardspring_production
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,789 AntiEntropyService.java 
 (line 652) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] new session: will 
 sync /X.X.X.190, /X.X.X.43, /X.X.X.56 on range 
 (1808575600,42535295865117307932921825930779602032] for 
 keyspace_production.[comma separated list of CFs]
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,790 AntiEntropyService.java 
 (line 858) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] requesting merkle 
 trees for BusinessConnectionIndicesEntries (to [/X.X.X.43, /X.X.X.56, 
 /X.X.X.190])
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,086 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.43
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,147 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.56
 Please advise. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5504) Eternal iteration when using newer hadoop version due to next() call and empty key value

[
https://issues.apache.org/jira/browse/CASSANDRA-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-5504:
--

Attachment: 5504-v3.txt

Thanks for the patch, Oleksandr.

It looks to me like the root of the problem is that
{{key.put(this.getCurrentKey())}} destructively modifies currentKey. Attached
is a patch to duplicate the buffer first.

This has the added benefit that we don't have to impose any overhead on the new
mapreduce api to solve this problem in the old mapred one.

Eternal iteration when using newer hadoop version due to next() call and
empty key value

Key: CASSANDRA-5504
URL: https://issues.apache.org/jira/browse/CASSANDRA-5504
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.2.3
Reporter: Oleksandr Petrov
Priority: Critical
Attachments: 5504-v3.txt, patch2.diff, patch.diff

Currently, when using newer hadoop versions, due to the call to
next(ByteBuffer key, SortedMapByteBuffer, IColumn value)
within ColumnFamilyRecordReader, because `key.clear();` is called, key is
emptied. That causes the StaticRowIterator and WideRowIterator to glitch,
namely, when Iterables.getLast(rows).key is called, key is already empty.
This will cause Hadoop to request the same range again and again all the time.
Please see the attached patch/diff, it simply adds lastRowKey (ByteBuffer)
and saves it for the next iteration along with all the rows, this allows
query for the next range to be fully correct.
This patch is branched from 1.2.3 version.
Tested against Cassandra 1.2.3, with Hadoop 1.0.3, 1.0.4 and 0.20.2

[jira] [Updated] (CASSANDRA-5504) Eternal iteration when using newer hadoop version due to next() call and empty key value


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5504:
--

 Priority: Minor  (was: Critical)
Affects Version/s: (was: 1.2.3)
   1.2.0
Fix Version/s: 1.2.5

While investigating whether this was also a problem in 1.1, I found that this 
was fixed for 1.1.7 in CASSANDRA-4834, with the same .duplicate() solution, but 
not merged forward.  I've applied this fix to the 1.2 branch.

 Eternal iteration when using newer hadoop version due to next() call and 
 empty key value
 

 Key: CASSANDRA-5504
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5504
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.0
Reporter: Oleksandr Petrov
Priority: Minor
 Fix For: 1.2.5

 Attachments: 5504-v3.txt, patch2.diff, patch.diff


 Currently, when using newer hadoop versions, due to the call to 
 next(ByteBuffer key, SortedMapByteBuffer, IColumn value)
 within ColumnFamilyRecordReader, because `key.clear();` is called, key is 
 emptied. That causes the StaticRowIterator and WideRowIterator to glitch, 
 namely, when Iterables.getLast(rows).key is called, key is already empty. 
 This will cause Hadoop to request the same range again and again all the time.
 Please see the attached patch/diff, it simply adds lastRowKey (ByteBuffer) 
 and saves it for the next iteration along with all the rows, this allows 
 query for the next range to be fully correct.
 This patch is branched from 1.2.3 version.
 Tested against Cassandra 1.2.3, with Hadoop 1.0.3, 1.0.4 and 0.20.2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/3] git commit: apply fix for CASSANDRA-4834 to the 1.2 branch. See also CASSANDRA-5504

Updated Branches:
  refs/heads/cassandra-1.2 68444f9b2 - 9793d6f32
  refs/heads/trunk 5d821a1ea - 4a104d904


apply fix for CASSANDRA-4834 to the 1.2 branch.  See also CASSANDRA-5504


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9793d6f3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9793d6f3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9793d6f3

Branch: refs/heads/cassandra-1.2
Commit: 9793d6f32de5aab8b45fbf870104617e02bf3183
Parents: 68444f9
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Apr 25 18:00:59 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Apr 25 18:00:59 2013 -0500

--
 .../cassandra/hadoop/ColumnFamilyRecordReader.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9793d6f3/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
index dfeacc3..a987519 100644
--- a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
@@ -461,7 +461,7 @@ public class ColumnFamilyRecordReader extends 
RecordReaderByteBuffer, SortedMap
 return endOfData();
 
 PairByteBuffer, SortedMapByteBuffer, IColumn next = 
wideColumns.next();
-lastColumn = next.right.values().iterator().next().name();
+lastColumn = 
next.right.values().iterator().next().name().duplicate();
 
 maybeIncreaseRowCounter(next);
 return next;
@@ -534,7 +534,7 @@ public class ColumnFamilyRecordReader extends 
RecordReaderByteBuffer, SortedMap
 if (this.nextKeyValue())
 {
 key.clear();
-key.put(this.getCurrentKey());
+key.put(this.getCurrentKey().duplicate());
 key.flip();
 
 value.clear();

[2/3] git commit: apply fix for CASSANDRA-4834 to the 1.2 branch. See also CASSANDRA-5504

apply fix for CASSANDRA-4834 to the 1.2 branch.  See also CASSANDRA-5504


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9793d6f3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9793d6f3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9793d6f3

Branch: refs/heads/trunk
Commit: 9793d6f32de5aab8b45fbf870104617e02bf3183
Parents: 68444f9
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Apr 25 18:00:59 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Apr 25 18:00:59 2013 -0500

--
 .../cassandra/hadoop/ColumnFamilyRecordReader.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9793d6f3/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
index dfeacc3..a987519 100644
--- a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
@@ -461,7 +461,7 @@ public class ColumnFamilyRecordReader extends 
RecordReaderByteBuffer, SortedMap
 return endOfData();
 
 PairByteBuffer, SortedMapByteBuffer, IColumn next = 
wideColumns.next();
-lastColumn = next.right.values().iterator().next().name();
+lastColumn = 
next.right.values().iterator().next().name().duplicate();
 
 maybeIncreaseRowCounter(next);
 return next;
@@ -534,7 +534,7 @@ public class ColumnFamilyRecordReader extends 
RecordReaderByteBuffer, SortedMap
 if (this.nextKeyValue())
 {
 key.clear();
-key.put(this.getCurrentKey());
+key.put(this.getCurrentKey().duplicate());
 key.flip();
 
 value.clear();

[3/3] git commit: merge from 1.2

merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a104d90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a104d90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a104d90

Branch: refs/heads/trunk
Commit: 4a104d904b809f071ca52b1d8ea3da9fa57ea392
Parents: 5d821a1 9793d6f
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Apr 25 18:01:38 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Apr 25 18:01:38 2013 -0500

--
 .../cassandra/hadoop/ColumnFamilyRecordReader.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a104d90/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
--
diff --cc src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
index 3764503,a987519..412e57d
--- a/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
+++ b/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java
@@@ -482,8 -460,8 +482,8 @@@ public class ColumnFamilyRecordReader e
  if (rows == null)
  return endOfData();
  
 -PairByteBuffer, SortedMapByteBuffer, IColumn next = 
wideColumns.next();
 +PairByteBuffer, SortedMapByteBuffer, Column next = 
wideColumns.next();
- lastColumn = next.right.values().iterator().next().name();
+ lastColumn = 
next.right.values().iterator().next().name().duplicate();
  
  maybeIncreaseRowCounter(next);
  return next;

[jira] [Updated] (CASSANDRA-5504) Eternal iteration when using older hadoop version due to next() call and empty key value


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5504:
--

Summary: Eternal iteration when using older hadoop version due to next() 
call and empty key value  (was: Eternal iteration when using newer hadoop 
version due to next() call and empty key value)

 Eternal iteration when using older hadoop version due to next() call and 
 empty key value
 

 Key: CASSANDRA-5504
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5504
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.0
Reporter: Oleksandr Petrov
Assignee: Oleksandr Petrov
Priority: Minor
 Fix For: 1.2.5

 Attachments: 5504-v3.txt, patch2.diff, patch.diff


 Currently, when using newer hadoop versions, due to the call to 
 next(ByteBuffer key, SortedMapByteBuffer, IColumn value)
 within ColumnFamilyRecordReader, because `key.clear();` is called, key is 
 emptied. That causes the StaticRowIterator and WideRowIterator to glitch, 
 namely, when Iterables.getLast(rows).key is called, key is already empty. 
 This will cause Hadoop to request the same range again and again all the time.
 Please see the attached patch/diff, it simply adds lastRowKey (ByteBuffer) 
 and saves it for the next iteration along with all the rows, this allows 
 query for the next range to be fully correct.
 This patch is branched from 1.2.3 version.
 Tested against Cassandra 1.2.3, with Hadoop 1.0.3, 1.0.4 and 0.20.2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5432) Repair Freeze/Gossip Invisibility Issues 1.2.4

2013-04-25 Thread Arya Goudarzi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642436#comment-13642436
 ] 

Arya Goudarzi commented on CASSANDRA-5432:
--

Sure, I should be able to get back to you either tonight or tomorrow.

 Repair Freeze/Gossip Invisibility Issues 1.2.4
 --

 Key: CASSANDRA-5432
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5432
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Ubuntu 10.04.1 LTS
 C* 1.2.3
 Sun Java 6 u43
 JNA Enabled
 Not using VNodes
Reporter: Arya Goudarzi
Assignee: Vijay
Priority: Critical
 Attachments: 0001-CASSANDRA-5432.patch


 Read comment 6. This description summarizes the repair issue only, but I 
 believe there is a bigger problem going on with networking as described on 
 that comment. 
 Since I have upgraded our sandbox cluster, I am unable to run repair on any 
 node and I am reaching our gc_grace seconds this weekend. Please help. So 
 far, I have tried the following suggestions:
 - nodetool scrub
 - offline scrub
 - running repair on each CF separately. Didn't matter. All got stuck the same 
 way.
 The repair command just gets stuck and the machine is idling. Only the 
 following logs are printed for repair job:
  INFO [Thread-42214] 2013-04-05 23:30:27,785 StorageService.java (line 2379) 
 Starting repair command #4, repairing 1 ranges for keyspace 
 cardspring_production
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,789 AntiEntropyService.java 
 (line 652) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] new session: will 
 sync /X.X.X.190, /X.X.X.43, /X.X.X.56 on range 
 (1808575600,42535295865117307932921825930779602032] for 
 keyspace_production.[comma separated list of CFs]
  INFO [AntiEntropySessions:7] 2013-04-05 23:30:27,790 AntiEntropyService.java 
 (line 858) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] requesting merkle 
 trees for BusinessConnectionIndicesEntries (to [/X.X.X.43, /X.X.X.56, 
 /X.X.X.190])
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,086 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.43
  INFO [AntiEntropyStage:1] 2013-04-05 23:30:28,147 AntiEntropyService.java 
 (line 214) [repair #cc5a9aa0-9e48-11e2-98ba-11bde7670242] Received merkle 
 tree for ColumnFamilyName from /X.X.X.56
 Please advise. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642487#comment-13642487
 ] 

Jonathan Ellis commented on CASSANDRA-5501:
---

Can you post your schema?

 Missing data on SELECT on secondary index 
 --

 Key: CASSANDRA-5501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5501
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.4
 Environment: linux ubuntu 12.04
Reporter: Marco Matarazzo
 Attachments: query_log.txt


 We have a 3 nodes cluster, and a keyspace with RF = 3.
 From cassandra-cli everything is fine (we actually never use it, I just 
 launched it for a check in this particular case).
 [default@goh_master] get agents where station_id = ascii(1110129);
 ---
 RowKey: 6c8efeb6-7209-11e2-890a-aacc0216
 = (column=, value=, timestamp=1364580868176000)
 = (column=character_points, value=, timestamp=136103068689)
 = (column=component_id, value=0, timestamp=1364580868176000)
 = (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, 
 timestamp=136103068689)
 = (column=entity_id, value=0, timestamp=1364580868176000)
 = (column=manufacturing, value=, timestamp=136103068689)
 = (column=model, value=55, timestamp=136103068689)
 = (column=name, value=Jenny Olifield, timestamp=136103068689)
 = (column=name_check, value=jenny_olifield, timestamp=136103068689)
 = (column=station_id, value=1110129, timestamp=1364580868176000)
 = (column=stats_intellect, value=8, timestamp=136103068689)
 = (column=stats_reflexes, value=8, timestamp=136103068689)
 = (column=stats_stamina, value=7, timestamp=136103068689)
 = (column=stats_technology, value=7, timestamp=136103068689)
 = (column=trading, value=, timestamp=136103068689)
 ---
 RowKey: dc413373-6b06-11e2-8943-aacc0216
 = (column=, value=, timestamp=136656818522)
 = (column=character_points, value=100, timestamp=1364580381651000)
 = (column=component_id, value=, timestamp=1364580381651000)
 = (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, 
 timestamp=1364580381651000)
 = (column=entity_id, value=0, timestamp=1364580381651000)
 = (column=manufacturing, value=, timestamp=1364580381651000)
 = (column=model, value=500018, timestamp=1364580381651000)
 = (column=name, value=Darren Matar, timestamp=1364580381651000)
 = (column=name_check, value=darren_matar, timestamp=1364580381651000)
 = (column=station_id, value=1110129, timestamp=1364580381651000)
 = (column=stats_intellect, value=10, timestamp=1364580381651000)
 = (column=stats_reflexes, value=10, timestamp=1364580381651000)
 = (column=stats_stamina, value=10, timestamp=1364580381651000)
 = (column=stats_technology, value=10, timestamp=1364580381651000)
 = (column=trading, value=1, timestamp=136656818522)
 ---
 RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201
 = (column=, value=, timestamp=1364828039093000)
 = (column=character_points, value=, timestamp=136103068676)
 = (column=component_id, value=0, timestamp=1364828039093000)
 = (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, 
 timestamp=136103068676)
 = (column=entity_id, value=0, timestamp=1364828039093000)
 = (column=manufacturing, value=1, timestamp=1362517535613000)
 = (column=model, value=58, timestamp=136103068676)
 = (column=name, value=Tom Bishop, timestamp=136103068676)
 = (column=name_check, value=tom_bishop, timestamp=136103068676)
 = (column=station_id, value=1110129, timestamp=1364828039093000)
 = (column=stats_intellect, value=9, timestamp=136103068676)
 = (column=stats_reflexes, value=7, timestamp=136103068676)
 = (column=stats_stamina, value=5, timestamp=136103068676)
 = (column=stats_technology, value=9, timestamp=136103068676)
 = (column=trading, value=, timestamp=136103068676)
 ---
 RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa
 = (column=, value=, timestamp=1366568185096000)
 = (column=character_points, value=100, timestamp=1364580381537000)
 = (column=component_id, value=, timestamp=1364580381537000)
 = (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, 
 timestamp=1364580381537000)
 = (column=entity_id, value=0, timestamp=1364580381537000)
 = (column=manufacturing, value=, timestamp=1364580381537000)
 = (column=model, value=53, timestamp=1364580381537000)
 = (column=name, value=Andrea Len, timestamp=1364580381537000)
 = (column=name_check, value=andrea_len, timestamp=1364580381537000)
 = (column=station_id, value=1110129, timestamp=1364580381537000)
 = (column=stats_intellect, value=10, timestamp=1364580381537000)
 = (column=stats_reflexes, value=10, timestamp=1364580381537000)
 = (column=stats_stamina, value=10,

[jira] [Comment Edited] (CASSANDRA-5501) Missing data on SELECT on secondary index


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642487#comment-13642487
 ] 

Jonathan Ellis edited comment on CASSANDRA-5501 at 4/26/13 1:35 AM:


Can you post this table's schema?

  was (Author: jbellis):
Can you post your schema?
  
 Missing data on SELECT on secondary index 
 --

 Key: CASSANDRA-5501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5501
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.4
 Environment: linux ubuntu 12.04
Reporter: Marco Matarazzo
 Attachments: query_log.txt


 We have a 3 nodes cluster, and a keyspace with RF = 3.
 From cassandra-cli everything is fine (we actually never use it, I just 
 launched it for a check in this particular case).
 [default@goh_master] get agents where station_id = ascii(1110129);
 ---
 RowKey: 6c8efeb6-7209-11e2-890a-aacc0216
 = (column=, value=, timestamp=1364580868176000)
 = (column=character_points, value=, timestamp=136103068689)
 = (column=component_id, value=0, timestamp=1364580868176000)
 = (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, 
 timestamp=136103068689)
 = (column=entity_id, value=0, timestamp=1364580868176000)
 = (column=manufacturing, value=, timestamp=136103068689)
 = (column=model, value=55, timestamp=136103068689)
 = (column=name, value=Jenny Olifield, timestamp=136103068689)
 = (column=name_check, value=jenny_olifield, timestamp=136103068689)
 = (column=station_id, value=1110129, timestamp=1364580868176000)
 = (column=stats_intellect, value=8, timestamp=136103068689)
 = (column=stats_reflexes, value=8, timestamp=136103068689)
 = (column=stats_stamina, value=7, timestamp=136103068689)
 = (column=stats_technology, value=7, timestamp=136103068689)
 = (column=trading, value=, timestamp=136103068689)
 ---
 RowKey: dc413373-6b06-11e2-8943-aacc0216
 = (column=, value=, timestamp=136656818522)
 = (column=character_points, value=100, timestamp=1364580381651000)
 = (column=component_id, value=, timestamp=1364580381651000)
 = (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, 
 timestamp=1364580381651000)
 = (column=entity_id, value=0, timestamp=1364580381651000)
 = (column=manufacturing, value=, timestamp=1364580381651000)
 = (column=model, value=500018, timestamp=1364580381651000)
 = (column=name, value=Darren Matar, timestamp=1364580381651000)
 = (column=name_check, value=darren_matar, timestamp=1364580381651000)
 = (column=station_id, value=1110129, timestamp=1364580381651000)
 = (column=stats_intellect, value=10, timestamp=1364580381651000)
 = (column=stats_reflexes, value=10, timestamp=1364580381651000)
 = (column=stats_stamina, value=10, timestamp=1364580381651000)
 = (column=stats_technology, value=10, timestamp=1364580381651000)
 = (column=trading, value=1, timestamp=136656818522)
 ---
 RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201
 = (column=, value=, timestamp=1364828039093000)
 = (column=character_points, value=, timestamp=136103068676)
 = (column=component_id, value=0, timestamp=1364828039093000)
 = (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, 
 timestamp=136103068676)
 = (column=entity_id, value=0, timestamp=1364828039093000)
 = (column=manufacturing, value=1, timestamp=1362517535613000)
 = (column=model, value=58, timestamp=136103068676)
 = (column=name, value=Tom Bishop, timestamp=136103068676)
 = (column=name_check, value=tom_bishop, timestamp=136103068676)
 = (column=station_id, value=1110129, timestamp=1364828039093000)
 = (column=stats_intellect, value=9, timestamp=136103068676)
 = (column=stats_reflexes, value=7, timestamp=136103068676)
 = (column=stats_stamina, value=5, timestamp=136103068676)
 = (column=stats_technology, value=9, timestamp=136103068676)
 = (column=trading, value=, timestamp=136103068676)
 ---
 RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa
 = (column=, value=, timestamp=1366568185096000)
 = (column=character_points, value=100, timestamp=1364580381537000)
 = (column=component_id, value=, timestamp=1364580381537000)
 = (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, 
 timestamp=1364580381537000)
 = (column=entity_id, value=0, timestamp=1364580381537000)
 = (column=manufacturing, value=, timestamp=1364580381537000)
 = (column=model, value=53, timestamp=1364580381537000)
 = (column=name, value=Andrea Len, timestamp=1364580381537000)
 = (column=name_check, value=andrea_len, timestamp=1364580381537000)
 = (column=station_id, value=1110129, timestamp=1364580381537000)
 = (column=stats_intellect, value=10,

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642517#comment-13642517
]

Jonathan Ellis commented on CASSANDRA-5417:
---

What if we took the approach from CASSANDRA-674 /
http://wiki.apache.org/cassandra/FileFormatDesignDoc, and had an explicit
parent atom instead of implicit via prefix sharing? Seems like that matches
better what we'd want to do with the actual bits on disk.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

CompositeType happens to be very useful and is now widely used: CQL3 heavily
rely on it, and super columns are now using it too internally. Besides,
CompositeType has been advised as a replacement of super columns on the
thrift side for a while, so it's safe to assume that it's generally used
there too.
CompositeType has initially been introduced as just another AbstractType.
Meaning that the storage engine has no nothing whatsoever of composites
being, well, composite. This has the following drawbacks:
* Because internally a composite value is handled as just a ByteBuffer, we
end up doing a lot of extra work. Typically, each time we compare 2 composite
value, we end up deserializing the components (which, while it doesn't copy
data per-se because we just slice the global ByteBuffer, still waste some cpu
cycles and allocate a bunch of ByteBuffer objects). And since compare can be
called *a lot*, this is likely not negligible.
* This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive
use of composites, and since it gets backs ByteBuffer from the internal
columns, it always have to check if it's actually a compositeType or not, and
then split it and pick the different parts it needs. It's only an API
problem, but having things exposed as composites directly would definitively
make thinks cleaner. In particular, in most cases, CQL3 don't care whether it
has a composite with only one component or a non-really-composite value, but
we still always distinguishes both cases. Lastly, if we do expose composites
more directly internally, it's not a lot more work to internalize better
the different parts of the cell name that CQL3 uses (what's the clustering
key, what's the actuall CQL3 column name, what's the collection element),
making things cleaner. Last but not least, there is currently a bunch of
places where methods take a ByteBuffer as argument and it's hard to know
whether it expects a cell name or a CQL3 column name. This is pretty error
prone.
* It makes it hard (or impossible) to do a number of performance
improvements. Consider CASSANDRA-4175, I'm not really sure how you can do it
properly (in memory) if cell names are just ByteBuffer (since CQL3 column
names are just one of the component in general). But we also miss
oportunities of sharing prefixes. If we were able to share prefixes of
composite names in memory we would 1) lower the memory footprint and 2)
potentially speed-up comparison (of the prefixes) by checking reference
equality first (also, doing prefix sharing on-disk, which is a separate
concern btw, might be easier to do if we do prefix sharing in memory).
So I suggest pushing CompositeType support inside the storage engine. What I
mean by that concretely would be change the internal {{Column.name}} from
ByteBuffer to some CellName type. A CellName would API-wise just be a list of
ByteBuffer. But in practice, we'd have a specific CellName implementation for
not-really-composite names, and the truly composite implementation will allow
some prefix sharing. From an external API however, nothing would change, we
would pack the composite as usual before sending it back to the client, but
at least internally, comparison won't have to deserialize the components
every time, and CQL3 code will be cleaner.

[jira] [Created] (CASSANDRA-5516) java.lang.NullPointerException at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)

2013-04-25 Thread Jouni Kontinen (JIRA)

Jouni Kontinen created CASSANDRA-5516:
-

 Summary: java.lang.NullPointerException at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)
 Key: CASSANDRA-5516
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5516
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.2
 Environment: Linux XYZ 3.5.0-27-generic #46~precise1-Ubuntu SMP Tue 
Mar 26 19:33:21 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Jouni Kontinen


ERROR 13:42:10,467 Exception in thread Thread[ReadStage:12,5,main]
java.lang.RuntimeException: java.lang.NullPointerException
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:85)
at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:36)
at java.util.Collections.indexedBinarySearch(Unknown Source)
at java.util.Collections.binarySearch(Unknown Source)
at 
org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:482)
at 
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:755)
at 
org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:717)
at 
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:43)
at 
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:101)
at 
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68)
at 
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:275)
at 
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
at 
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1363)
at 
org.apache.cassandra.db.ColumnFamilyStore.getThroughCache(ColumnFamilyStore.java:1176)
at 
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1209)
at 
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1132)
at org.apache.cassandra.db.Table.getRow(Table.java:355)
at 
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
at 
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052)
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578)
... 3 more
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x7f367e28b14e, pid=7843, tid=139871526840064
#
# JRE version: 6.0_38-b05
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.13-b02 mixed mode linux-amd64 
compressed oops)
# Problematic frame:
# V  [libjvm.so+0x35514e]  ciObjectFactory::find_non_perm(oopDesc*)+0x4e
#
# An error report file with more information is saved as:
# /var/lib/cassandra/hs_err_1365567530.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---

That caused Cassandra to shutdown. I believe it corrupted my data as I started 
getting this: 

java.lang.RuntimeException: 
org.apache.cassandra.io.sstable.CorruptSSTableException: 
org.apache.cassandra.io.compress.CorruptBlockException



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-5516) java.lang.NullPointerException at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)