date:20130626


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693680#comment-13693680
 ] 

Pavel Yaskevich commented on CASSANDRA-5661:


What is LCS?

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4775) Counters 2.0

2013-06-26 Thread Colin B. (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693681#comment-13693681
]

Colin B. commented on CASSANDRA-4775:
-

Would anyone be interested in a type of counter that is about 99% correct, but
not exact?

The Hyperloglog cardinality estimation algorithm would be fairly straight
forward to implement inside Cassandra. It estimates the number of distinct
elements in a set. One way to use it as a counter is to have a two sets of
timeuuids, A and R. Each time you want to increment the counter add a timeuuid
to the A set, each time you want to decrement add a timeuuid to the R set.
Count is the count in A minus the count in R. Re-adding the same item
(timeuuid) to a set is idempotent. A read would need to access a constant
amount of internal data and the internal data is a good fit for Cassandra's
method of merging distributed writes.

A description of the Hyperloglog algorithm is available here:
http://blog.aggregateknowledge.com/2012/10/25/sketch-of-the-day-hyperloglog-cornerstone-of-a-big-data-infrastructure/

Counters 2.0

Key: CASSANDRA-4775
URL: https://issues.apache.org/jira/browse/CASSANDRA-4775
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Arya Goudarzi
Assignee: Aleksey Yeschenko
Labels: counters
Fix For: 2.1

The existing partitioned counters remain a source of frustration for most
users almost two years after being introduced. The remaining problems are
inherent in the design, not something that can be fixed given enough
time/eyeballs.
Ideally a solution would give us
- similar performance
- less special cases in the code
- potential for a retry mechanism

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5683) Write timeout in multi-dc environment

2013-06-26 Thread Boole Guo (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693826#comment-13693826
 ] 

Boole Guo commented on CASSANDRA-5683:
--

Can we add a nodetool command for modify rpctime? 

 Write timeout in multi-dc environment 
 --

 Key: CASSANDRA-5683
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5683
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.1.9
 Environment: apache cassandra 1.1.9
Reporter: Boole Guo
  Labels: write

 When writing in mutil-dc environment, there are many timeout exception. As I 
 know, this version have do good to reduce network bandwidth. Can we assign 
 rpc timeout when writing like consistenylevel?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5701) Apache.Cassandra.Cassandar.get_count will disconnect but not throw InvalidRequestException when column family is not exit.

yuemaoxing created CASSANDRA-5701:
-

 Summary: Apache.Cassandra.Cassandar.get_count will disconnect but 
not throw InvalidRequestException when column family is not exit.
 Key: CASSANDRA-5701
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5701
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.2.4
 Environment: cassandra server : 1.2.4
and cassandra1.2.5 has same bug.
client : C# client
Reporter: yuemaoxing


When I use get_count interface which is defined in Cassandra.thrift, the 
Cassandra Server(1.2.4) close the conncet from Client when column family is not 
exit in that keyspace but not throw InvalidRequestException. 

It seemed the get_count method in cassandra.thrift.CassandraServer.java did not 
validate parameters(ThriftValidation.validateColumnFamily) in this method.


system.log:
ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
638) Unexpected exception while invoking!
java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
at 
org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
at 
org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


Please check this bug, thanks!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5701) Apache.Cassandra.Cassandar.get_count will disconnect but not throw InvalidRequestException when column family is not exist.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yuemaoxing updated CASSANDRA-5701:
--

Description: 
When I use get_count interface which is defined in Cassandra.thrift, the 
Cassandra Server(1.2.4) close the connect from Client when column family is not 
exist in that keyspace but not throw InvalidRequestException. 

It seemed the get_count method in cassandra.thrift.CassandraServer.java did not 
validate parameters(ThriftValidation.validateColumnFamily) in this method.


system.log:
ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
638) Unexpected exception while invoking!
java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
at 
org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
at 
org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


Please check this bug, thanks!

  was:
When I use get_count interface which is defined in Cassandra.thrift, the 
Cassandra Server(1.2.4) close the conncet from Client when column family is not 
exit in that keyspace but not throw InvalidRequestException. 

It seemed the get_count method in cassandra.thrift.CassandraServer.java did not 
validate parameters(ThriftValidation.validateColumnFamily) in this method.


system.log:
ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
638) Unexpected exception while invoking!
java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
at 
org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
at 
org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


Please check this bug, thanks!

Summary: Apache.Cassandra.Cassandar.get_count will disconnect but not 
throw InvalidRequestException when column family is not exist.  (was: 
Apache.Cassandra.Cassandar.get_count will disconnect but not throw 
InvalidRequestException when column family is not exit.)

 Apache.Cassandra.Cassandar.get_count will disconnect but not throw 
 InvalidRequestException when column family is not exist.
 ---

 Key: CASSANDRA-5701
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5701
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.2.4
 Environment: cassandra server : 1.2.4
 and cassandra1.2.5 has same bug.
 client : C# client
Reporter: yuemaoxing
  Labels: patch
   Original Estimate: 24h
  Remaining Estimate: 24h

 When I use get_count interface which is defined in Cassandra.thrift, the 
 Cassandra Server(1.2.4) close the connect from Client when column family is 
 not exist in that keyspace but not throw InvalidRequestException. 
 It seemed the get_count method in cassandra.thrift.CassandraServer.java did 
 not validate parameters(ThriftValidation.validateColumnFamily) in this method.
 system.log:
 ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
 638) Unexpected exception while invoking!
 java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
   at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)

[jira] [Updated] (CASSANDRA-5701) Apache.Cassandra.Cassandra.get_count will disconnect but not throw InvalidRequestException when column family is not exist.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yuemaoxing updated CASSANDRA-5701:
--

Summary: Apache.Cassandra.Cassandra.get_count will disconnect but not throw 
InvalidRequestException when column family is not exist.  (was: 
Apache.Cassandra.Cassandar.get_count will disconnect but not throw 
InvalidRequestException when column family is not exist.)

 Apache.Cassandra.Cassandra.get_count will disconnect but not throw 
 InvalidRequestException when column family is not exist.
 ---

 Key: CASSANDRA-5701
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5701
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.2.4
 Environment: cassandra server : 1.2.4
 and cassandra1.2.5 has same bug.
 client : C# client
Reporter: yuemaoxing
  Labels: patch
   Original Estimate: 24h
  Remaining Estimate: 24h

 When I use get_count interface which is defined in Cassandra.thrift, the 
 Cassandra Server(1.2.4) close the connect from Client when column family is 
 not exist in that keyspace but not throw InvalidRequestException. 
 It seemed the get_count method in cassandra.thrift.CassandraServer.java did 
 not validate parameters(ThriftValidation.validateColumnFamily) in this method.
 system.log:
 ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
 638) Unexpected exception while invoking!
 java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
   at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
   at 
 org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
   at 
 org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
   at 
 org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
 Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)
 Please check this bug, thanks!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5701) Apache.Cassandra.Cassandra.get_count will disconnect but not throw InvalidRequestException when column family is not exist.


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yuemaoxing updated CASSANDRA-5701:
--

Description: 
When I use get_count interface which is defined in Cassandra.thrift, the 
Cassandra Server(1.2.4) close the connection from Client when column family is 
not exist in that keyspace but not throw InvalidRequestException. 

It seemed the get_count method in cassandra.thrift.CassandraServer.java did not 
validate parameters(ThriftValidation.validateColumnFamily) in this method.


system.log:
ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
638) Unexpected exception while invoking!
java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
at 
org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
at 
org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


Please check this bug, thanks!

  was:
When I use get_count interface which is defined in Cassandra.thrift, the 
Cassandra Server(1.2.4) close the connect from Client when column family is not 
exist in that keyspace but not throw InvalidRequestException. 

It seemed the get_count method in cassandra.thrift.CassandraServer.java did not 
validate parameters(ThriftValidation.validateColumnFamily) in this method.


system.log:
ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
638) Unexpected exception while invoking!
java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
at 
org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
at 
org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3369)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at 
org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632)
at 
org.apache.cassandra.thrift.CustomTHsHaServer$Invocation.run(CustomTHsHaServer.java:109)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


Please check this bug, thanks!


 Apache.Cassandra.Cassandra.get_count will disconnect but not throw 
 InvalidRequestException when column family is not exist.
 ---

 Key: CASSANDRA-5701
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5701
 Project: Cassandra
  Issue Type: Bug
  Components: API
Affects Versions: 1.2.4
 Environment: cassandra server : 1.2.4
 and cassandra1.2.5 has same bug.
 client : C# client
Reporter: yuemaoxing
  Labels: patch
   Original Estimate: 24h
  Remaining Estimate: 24h

 When I use get_count interface which is defined in Cassandra.thrift, the 
 Cassandra Server(1.2.4) close the connection from Client when column family 
 is not exist in that keyspace but not throw InvalidRequestException. 
 It seemed the get_count method in cassandra.thrift.CassandraServer.java did 
 not validate parameters(ThriftValidation.validateColumnFamily) in this method.
 system.log:
 ERROR [RPC-Thread:3373] 2013-06-26 14:23:09,264 TNonblockingServer.java (line 
 638) Unexpected exception while invoking!
 java.lang.IllegalArgumentException: Unknown table/cf pair (Keyspace1.Standard)
   at org.apache.cassandra.db.Table.getColumnFamilyStore(Table.java:165)
   at 
 org.apache.cassandra.thrift.CassandraServer.get_count(CassandraServer.java:471)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$get_count.getResult(Cassandra.java:3381)
   at

[jira] [Created] (CASSANDRA-5702) ALTER RENAME is broken in trunk

Sylvain Lebresne created CASSANDRA-5702:
---

 Summary: ALTER RENAME is broken in trunk
 Key: CASSANDRA-5702
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5702
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.0 beta 1


CASSANDRA-5125 has broken {{ALTER RENAME}} when the column is a default alias 
(for thrift column families where the PK columns haven't been renamed yet).

The problem is basically that while we assign default aliases to PK columns 
when they don't have one, we currently fake those default aliases and do not 
persist them. Concretely, CFDefinition is aware of them, but CFMetaData is not, 
which break renaming post CASSANDRA-5125.

We could fix rename punctually, but there is another related problem: for the 
same reason, if you try to create an index on a column that is a non-renamed 
default alias, this doesn't work with the arguably confusing message No column 
definition found for column X. Here again, we could fix it punctually, but it 
starts to sound like we need a more general fix.

So I suggest stopping to fake those default aliases, but instead to just 
create real aliases (that are known of CFMetaData and persisted in the 
schema) when there is none. After all, from a user point of view, why should a 
default column name be any special. And on top of fixing the issues above, this 
also:
# fix CASSANDRA-5489 in a somewhat simpler way
# makes it easier for clients reading the schema CFs. They won't to infer the 
default aliases anymore.

The only theoretical downside is that we lose the information that a given CQL3 
column name is one assigned by default versus one set up by the user, but given 
the user can rename those column names anyway, not sure this matters in any way.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

2013-06-26 Thread Jeremiah Jordan (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693876#comment-13693876
]

Jeremiah Jordan commented on CASSANDRA-5661:

[~xedin] LCS == LeveledCompactionStrategy. While in theory I like the idea of
the searchers in most cases expiring quicker with timers, in practice since
these are on the heap, and LCS defaults to only 5 MB files per levels, you can
have A LOT of sstables, 200 per GB... Which as [~jbellis] mentioned makes # of
sstables * concurrent readers a big problem, which was the problem hit above.
So we really need to bound memory usage with a hard cap, not the fuzzy cap of
how many can I open in X seconds.

1k reqs/sec hitting a 4 level deep LCS CF could mean 4k Reader's (~500 MB)
created per second. Now that I say that out loud, I almost think we should go
back to not caching these at all so they always just get recycled in young gen
and never have a chance to hit old gen. I guess it could be a config parameter
which defaults on for STCS and off for LCS. Since STCS work loads have no
where near as many sstables.

Discard pooled readers for cold data

Key: CASSANDRA-5661
URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
Fix For: 1.2.7

Reader pooling was introduced in CASSANDRA-4942 but pooled
RandomAccessReaders are never cleaned up until the SSTableReader is closed.
So memory use is the worst case simultaneous RAR we had open for this file,
forever.
We should introduce a global limit on how much memory to use for RAR, and
evict old ones.

[jira] [Commented] (CASSANDRA-5702) ALTER RENAME is broken in trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693902#comment-13693902
]

Aleksey Yeschenko commented on CASSANDRA-5702:
--

Thought about this as well (in the context of CASSANDRA-5489) and came to the
same conclusion.

ALTER RENAME is broken in trunk
---

Key: CASSANDRA-5702
URL: https://issues.apache.org/jira/browse/CASSANDRA-5702
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0 beta 1

CASSANDRA-5125 has broken {{ALTER RENAME}} when the column is a default alias
(for thrift column families where the PK columns haven't been renamed yet).
The problem is basically that while we assign default aliases to PK columns
when they don't have one, we currently fake those default aliases and do
not persist them. Concretely, CFDefinition is aware of them, but CFMetaData
is not, which break renaming post CASSANDRA-5125.
We could fix rename punctually, but there is another related problem: for the
same reason, if you try to create an index on a column that is a non-renamed
default alias, this doesn't work with the arguably confusing message No
column definition found for column X. Here again, we could fix it
punctually, but it starts to sound like we need a more general fix.
So I suggest stopping to fake those default aliases, but instead to just
create real aliases (that are known of CFMetaData and persisted in the
schema) when there is none. After all, from a user point of view, why should
a default column name be any special. And on top of fixing the issues above,
this also:
# fix CASSANDRA-5489 in a somewhat simpler way
# makes it easier for clients reading the schema CFs. They won't to infer the
default aliases anymore.
The only theoretical downside is that we lose the information that a given
CQL3 column name is one assigned by default versus one set up by the user,
but given the user can rename those column names anyway, not sure this
matters in any way.

[jira] [Created] (CASSANDRA-5703) IndexSummary and empty keys

2013-06-26 Thread Fabien Rousseau (JIRA)

Fabien Rousseau created CASSANDRA-5703:
--

 Summary: IndexSummary and empty keys
 Key: CASSANDRA-5703
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5703
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Fabien Rousseau


We just migrated one of our dev cluster to 1.2.6 and got this Exception for an 
SSTable after restarting nodes :

2013-06-26 11:18:46,316 [SSTableBatchOpen:1] ERROR CassandraDaemon - Exception 
in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.AssertionError
at 
org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
at 
org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)


The assertion :
https://github.com/apache/cassandra/blob/9851b73fc3bbaf61ec80b54c64483bcf56aaa396/src/java/org/apache/cassandra/utils/ByteBufferUtil.java#L401

should probably be assert length = 0;

In a broader context, ByteBufferUtil.readBytes(...) could still be used to read 
0 bytes without throwing an assertion

Note : This happens because the SSTable contains an empty rowkey. We're using 
StorageProxy directly, thus skipping validation, but it appears that other 
people hit this problem too : 
http://www.mail-archive.com/user@cassandra.apache.org/msg30123.html

Also note that data contained in this SSTable is not available because the 
assertion error is not catched.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5683) Write timeout in multi-dc environment


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693954#comment-13693954
 ] 

Jonathan Ellis commented on CASSANDRA-5683:
---

Yes, that would be reasonable.

 Write timeout in multi-dc environment 
 --

 Key: CASSANDRA-5683
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5683
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.1.9
 Environment: apache cassandra 1.1.9
Reporter: Boole Guo
  Labels: write

 When writing in mutil-dc environment, there are many timeout exception. As I 
 know, this version have do good to reduce network bandwidth. Can we assign 
 rpc timeout when writing like consistenylevel?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5702) ALTER RENAME is broken in trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aleksey Yeschenko updated CASSANDRA-5702:
-

Reviewer: iamaleksey

ALTER RENAME is broken in trunk
---

Key: CASSANDRA-5702
URL: https://issues.apache.org/jira/browse/CASSANDRA-5702
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0 beta 1

Attachments: 5702.txt

[jira] [Updated] (CASSANDRA-5702) ALTER RENAME is broken in trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sylvain Lebresne updated CASSANDRA-5702:

Attachment: 5702.txt

Attaching patch for this. Note that persisting those default columns implied to
only rebuild CQL3 metadata once the CFMetaData is basically complete, hence the
patch moves the calls to said rebuild a bit (it's a good thing anyway, we were
calling that rebuild way too often). Also:
* the patch fixes a small bug in CFMetaData.isThriftCompatible(). It was
excluding way too much stuff post CASSANDRA-5125.
* this happen to lift the limitation introduced by CASSANDRA-5531 (not a big
deal but not a bad thing either).
* I've also rewritten CFMetaData.isDense(). I don't think that was strictly
necessary, but while trying to check it wasn't broken by the patch it felt it
could be simplified/clarified.

ALTER RENAME is broken in trunk
---

Key: CASSANDRA-5702
URL: https://issues.apache.org/jira/browse/CASSANDRA-5702
Project: Cassandra
Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0 beta 1

Attachments: 5702.txt

[jira] [Created] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false

Christian Spriegel created CASSANDRA-5704:
-

 Summary: Truncate flushes to disk again in 1.2, even with 
durable_writes=false
 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor


I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my JUnit 
tests slow again, due to a blocking-flush in saveTruncationRecord().

With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153


My proposal is to make saveTruncationRecord() only flush when durableWrites are 
enabled.

My assumption is that if somebody turn off durable writes then he does not mind 
if truncate is not guaranteed to be durable either.


I successfully tested the following patch with my testsuite. Its as fast as it 
was with 1.1 (maybe even faster!):
{code}
@@ -186,5 +186,8 @@ public class SystemTable
 String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
WHERE key = '%s';
 processInternal(String.format(req, LOCAL_CF, truncationAsMapEntry(cfs, 
truncatedAt, position), LOCAL_KEY));
-forceBlockingFlush(LOCAL_CF);
+
+KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
+if (ksm.durableWrites) // flush only when durable_writes are enabled
+forceBlockingFlush(LOCAL_CF);
 }
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Spriegel updated CASSANDRA-5704:
--

Attachment: CASSANDRA_5704_V1.patch

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Attachments: CASSANDRA_5704_V1.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Spriegel updated CASSANDRA-5704:
--

Attachment: (was: CASSANDRA_5704_V1.patch)

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor

 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christian Spriegel updated CASSANDRA-5704:
--

Attachment: CASSANDRA_5704_V1_trunk.patch
CASSANDRA_5704_V1_1_2.patch

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Git Push Summary

Updated Tags:  refs/tags/1.2.6-tentative [deleted] a6ca5d496

Git Push Summary

Updated Tags:  refs/tags/cassandra-1.2.6 [created] 55004ecec

[jira] [Commented] (CASSANDRA-5700) compact storage value restriction message confusing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693988#comment-13693988
 ] 

Jonathan Ellis commented on CASSANDRA-5700:
---

How about this, then?

Predicates on the non-primary-key column (user) of a COMPACT table are not yet 
supported

 compact storage value restriction message confusing
 ---

 Key: CASSANDRA-5700
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5700
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.2.5
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 1.2.7

 Attachments: 5700.txt


 i have a compact storage value column name (user) the same as another column 
 family name (user) and was getting the error
 Restricting the value of a compact CF (user) is not supported
 which was very confusing
 changed message to
 Restricting the value (user) of a compact CF is not supported
 (tackling the big problems)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693990#comment-13693990
 ] 

Jonathan Ellis commented on CASSANDRA-5661:
---

I'm not as pessimistic -- I think pooling will still be useful for most 
workloads -- but if we add a configurable ceiling on memory use, we could 
certainly add that zero = no pooling at all.

Would also be useful to track the pool hit rate.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693995#comment-13693995
 ] 

Jonathan Ellis commented on CASSANDRA-5704:
---

Hmm.  I'm not sure I'm convinced that we should not save the truncation record 
when durable writes are disabled.  Durable writes means I'm okay if you lose 
the data I'm about to write in case of power failure, but not flushing the 
truncate means Data that was supposed to be gone, could reappear which feels 
semantically different to me.

WDYT [~brandon.williams] [~tjake]?

(I note that for production use, CASSANDRA-4906 made the biggest difference 
since now we only need to flush the table being truncated instead of everyone.)

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13693997#comment-13693997
 ] 

Brandon Williams commented on CASSANDRA-5704:
-

bq. Data that was supposed to be gone, could reappear which feels 
semantically different to me.

I tend to agree.  Certainly if the biggest problem here is tests are slow, it's 
not worth the risk just to speed them up.  We could do something like add a 
flag if we need this.

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-5703) IndexSummary and empty keys


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5703.
---

Resolution: Invalid

Empty row keys are indeed invalid.

 IndexSummary and empty keys
 ---

 Key: CASSANDRA-5703
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5703
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Fabien Rousseau

 We just migrated one of our dev cluster to 1.2.6 and got this Exception for 
 an SSTable after restarting nodes :
 2013-06-26 11:18:46,316 [SSTableBatchOpen:1] ERROR CassandraDaemon - 
 Exception in thread Thread[SSTableBatchOpen:1,5,main]
 java.lang.AssertionError
 at 
 org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:401)
 at 
 org.apache.cassandra.io.sstable.IndexSummary$IndexSummarySerializer.deserialize(IndexSummary.java:124)
 The assertion :
 https://github.com/apache/cassandra/blob/9851b73fc3bbaf61ec80b54c64483bcf56aaa396/src/java/org/apache/cassandra/utils/ByteBufferUtil.java#L401
 should probably be assert length = 0;
 In a broader context, ByteBufferUtil.readBytes(...) could still be used to 
 read 0 bytes without throwing an assertion
 Note : This happens because the SSTable contains an empty rowkey. We're using 
 StorageProxy directly, thus skipping validation, but it appears that other 
 people hit this problem too : 
 http://www.mail-archive.com/user@cassandra.apache.org/msg30123.html
 Also note that data contained in this SSTable is not available because the 
 assertion error is not catched.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false

2013-06-26 Thread T Jake Luciani (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694013#comment-13694013
 ] 

T Jake Luciani commented on CASSANDRA-5704:
---

I agree, perhaps add sone test.mode property that skips it for unit tests?

 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1496950 - in /cassandra/site: publish/doc/cql3/CQL.html publish/download/index.html publish/index.html src/settings.py

Author: slebresne
Date: Wed Jun 26 14:51:33 2013
New Revision: 1496950

URL: http://svn.apache.org/r1496950
Log:
Update website for 1.2.6 release

Modified:
cassandra/site/publish/doc/cql3/CQL.html
cassandra/site/publish/download/index.html
cassandra/site/publish/index.html
cassandra/site/src/settings.py

Modified: cassandra/site/publish/doc/cql3/CQL.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL.html?rev=1496950r1=1496949r2=1496950view=diff
==
--- cassandra/site/publish/doc/cql3/CQL.html (original)
+++ cassandra/site/publish/doc/cql3/CQL.html Wed Jun 26 14:51:33 2013
@@ -1,4 +1,4 @@
-?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD 
XHTML 1.0 Transitional//EN 
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html 
xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type 
content=text/html; charset=utf-8/titleCQL/title/headbodyplink 
rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 
id=CassandraQueryLanguageCQLv3.0.3Cassandra Query Language (CQL) 
v3.0.3/h1span id=tableOfContentsol style=list-style: none;lia 
href=CQL.html#CassandraQueryLanguageCQLv3.0.3Cassandra Query Language (CQL) 
v3.0.3/aol style=list-style: none;lia href=CQL.html#CQLSyntaxCQL 
Syntax/aol style=list-style: none;lia 
href=CQL.html#PreamblePreamble/a/lilia 
href=CQL.html#ConventionsConventions/a/lilia 
href=CQL.html#identifiersIdentifiers and keywords/a/lilia 
href=CQL.html#constantsConstants/a/lilia 
href=CQL.html#CommentsComment
 s/a/lilia href=CQL.html#statementsStatements/a/lilia 
href=CQL.html#preparedStatementPrepared Statement/a/li/ol/lilia 
href=CQL.html#dataDefinitionData Definition/aol style=list-style: 
none;lia href=CQL.html#createKeyspaceStmtCREATE KEYSPACE/a/lilia 
href=CQL.html#useStmtUSE/a/lilia 
href=CQL.html#alterKeyspaceStmtALTER KEYSPACE/a/lilia 
href=CQL.html#dropKeyspaceStmtDROP KEYSPACE/a/lilia 
href=CQL.html#createTableStmtCREATE TABLE/a/lilia 
href=CQL.html#alterTableStmtALTER TABLE/a/lilia 
href=CQL.html#dropTableStmtDROP TABLE/a/lilia 
href=CQL.html#truncateStmtTRUNCATE/a/lilia 
href=CQL.html#createIndexStmtCREATE INDEX/a/lilia 
href=CQL.html#dropIndexStmtDROP INDEX/a/li/ol/lilia 
href=CQL.html#dataManipulationData Manipulation/aol style=list-style: 
none;lia href=CQL.html#insertStmtINSERT/a/lilia 
href=CQL.html#updateStmt
 UPDATE/a/lilia href=CQL.html#deleteStmtDELETE/a/lilia 
 href=CQL.html#batchStmtBATCH/a/li/ol/lilia 
 href=CQL.html#queriesQueries/aol style=list-style: none;lia 
 href=CQL.html#selectStmtSELECT/a/li/ol/lilia 
 href=CQL.html#typesData Types/aol style=list-style: none;lia 
 href=CQL.html#usingdatesWorking with dates/a/lilia 
 href=CQL.html#countersCounters/a/lilia 
 href=CQL.html#collectionsWorking with collections/a/li/ol/lilia 
 href=CQL.html#functionsFunctions/aol style=list-style: none;lia 
 href=CQL.html#tokenFunToken/a/lilia 
 href=CQL.html#timeuuidFunTimeuuid functions/a/lilia 
 href=CQL.html#blobFunBlob conversion functions/a/li/ol/lilia 
 href=CQL.html#appendixAAppendix A: CQL Keywords/a/lilia 
 href=CQL.html#changesChanges/aol style=list-style: none;lia 
 href=CQL.html#a3.0.33.0.3/a/lilia href=CQL.html#a3.0.23.0.
 2/a/lilia href=CQL.html#a3.0.13.0.1/a/li/ol/lilia 
href=CQL.html#VersioningVersioning/a/li/ol/li/ol/spanh2 
id=CQLSyntaxCQL Syntax/h2h3 id=PreamblePreamble/h3pThis document 
describes the Cassandra Query Language (CQL) version 3. CQL v3 is not backward 
compatible with CQL v2 and differs from it in numerous ways. Note that this 
document describes the last version of the languages. However, the a 
href=#changeschanges/a section provides the diff between the different 
versions of CQL v3./ppCQL v3 offers a model very close to SQL in the sense 
that data is put in emtables/em containing emrows/em of 
emcolumns/em. For that reason, when used in this document, these terms 
(tables, rows and columns) have the same definition than they have in SQL. But 
please note that as such, they do strongnot/strong refer to the concept of 
rows and columns found in the internal implementation of Cassandra and in the 
thrift and CQ
 L v2 API./ph3 id=ConventionsConventions/h3pTo aid in specifying the 
CQL syntax, we will use the following conventions in this 
document:/pulliLanguage rules will be given in a a 
href=http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form;BNF/a -like 
notation:/li/ulpre class=syntaxprelt;start ::= TERMINAL 
lt;non-terminal1 lt;non-terminal1
+?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD 
XHTML 1.0 Transitional//EN 
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html 
xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type 
content=text/html; charset=utf-8/titleCQL/title/headbodyplink 
rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 
id=CassandraQueryLanguageCQLv3.0.4Cassandra Query Language (CQL) 
v3.0.4/h1span id=tableOfContentsol

git commit: change SimpleCondition.signal to UOE patch by Mikhail Mazursky; reviewed by jbellis for CASSANDRA-5691

2013-06-26 Thread jbellis

Updated Branches:
  refs/heads/trunk 3455f1b76 - 3ec4ff5ed


change SimpleCondition.signal to UOE
patch by Mikhail Mazursky; reviewed by jbellis for CASSANDRA-5691


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3ec4ff5e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3ec4ff5e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3ec4ff5e

Branch: refs/heads/trunk
Commit: 3ec4ff5ed3f362fc77f25c31ae16afbb46030624
Parents: 3455f1b
Author: Jonathan Ellis jbel...@apache.org
Authored: Wed Jun 26 07:55:28 2013 -0700
Committer: Jonathan Ellis jbel...@apache.org
Committed: Wed Jun 26 07:55:28 2013 -0700

--
 .../compaction/ParallelCompactionIterable.java  | 26 +---
 .../service/AbstractWriteResponseHandler.java   |  2 +-
 .../apache/cassandra/service/ReadCallback.java  |  2 +-
 .../service/TruncateResponseHandler.java|  2 +-
 .../apache/cassandra/utils/SimpleCondition.java |  5 ++--
 5 files changed, 22 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3ec4ff5e/src/java/org/apache/cassandra/db/compaction/ParallelCompactionIterable.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/ParallelCompactionIterable.java 
b/src/java/org/apache/cassandra/db/compaction/ParallelCompactionIterable.java
index 1d380f6..a8baf7a 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/ParallelCompactionIterable.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/ParallelCompactionIterable.java
@@ -195,7 +195,7 @@ public class ParallelCompactionIterable extends 
AbstractCompactionIterable
 private class DeserializedColumnIterator implements OnDiskAtomIterator
 {
 private final Row row;
-private IteratorColumn iter;
+private final IteratorColumn iter;
 
 public DeserializedColumnIterator(Row row)
 {
@@ -236,7 +236,6 @@ public class ParallelCompactionIterable extends 
AbstractCompactionIterable
 {
 private final LinkedBlockingQueueRowContainer queue = new 
LinkedBlockingQueueRowContainer(1);
 private static final RowContainer finished = new RowContainer((Row) 
null);
-private Condition condition;
 private final ICompactionScanner scanner;
 
 public Deserializer(ICompactionScanner ssts, final int maxInMemorySize)
@@ -246,11 +245,14 @@ public class ParallelCompactionIterable extends 
AbstractCompactionIterable
 {
 protected void runMayThrow() throws Exception
 {
+SimpleCondition condition = null;
 while (true)
 {
 if (condition != null)
+{
 condition.await();
-
+condition = null;
+}
 if (!scanner.hasNext())
 {
 queue.put(finished);
@@ -260,13 +262,13 @@ public class ParallelCompactionIterable extends 
AbstractCompactionIterable
 SSTableIdentityIterator iter = 
(SSTableIdentityIterator) scanner.next();
 if (iter.dataSize  maxInMemorySize)
 {
-logger.debug(parallel lazy deserialize from  + 
iter.getPath());
+logger.debug(parallel lazy deserialize from {}, 
iter.getPath());
 condition = new SimpleCondition();
 queue.put(new RowContainer(new 
NotifyingSSTableIdentityIterator(iter, condition)));
 }
 else
 {
-logger.debug(parallel eager deserialize from  + 
iter.getPath());
+logger.debug(parallel eager deserialize from {}, 
iter.getPath());
 queue.put(new RowContainer(new Row(iter.getKey(), 
iter.getColumnFamilyWithColumns(ArrayBackedSortedColumns.factory;
 }
 }
@@ -301,9 +303,9 @@ public class ParallelCompactionIterable extends 
AbstractCompactionIterable
 private static class NotifyingSSTableIdentityIterator implements 
OnDiskAtomIterator
 {
 private final SSTableIdentityIterator wrapped;
-private final Condition condition;
+private final SimpleCondition condition;
 
-public NotifyingSSTableIdentityIterator(SSTableIdentityIterator 
wrapped, Condition condition)
+public NotifyingSSTableIdentityIterator(SSTableIdentityIterator 
wrapped, SimpleCondition condition)
 {
 this.wrapped = wrapped;

[jira] [Created] (CASSANDRA-5705) Disallow renaming an indexed column

Sylvain Lebresne created CASSANDRA-5705:
---

 Summary: Disallow renaming an indexed column
 Key: CASSANDRA-5705
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5705
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0 beta 1


We can now index CQL3 primary key parts (CASSANDRA-5125) but also allow to 
rename such columns. Unfortunately, since 2ndary index CF names currently 
depend on the name of the column they index, renaming an indexed column 
basically ends up dropping the index.

For now, the simplest solution is probably to just forbid renaming when the 
column is 2ndary indexed. If you really want to rename, you can always drop the 
index and recreate it after the rename.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5705) Disallow renaming an indexed column


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-5705:


Attachment: 5705.txt

Attaching trivial patch.

 Disallow renaming an indexed column
 ---

 Key: CASSANDRA-5705
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5705
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0 beta 1

 Attachments: 5705.txt


 We can now index CQL3 primary key parts (CASSANDRA-5125) but also allow to 
 rename such columns. Unfortunately, since 2ndary index CF names currently 
 depend on the name of the column they index, renaming an indexed column 
 basically ends up dropping the index.
 For now, the simplest solution is probably to just forbid renaming when the 
 column is 2ndary indexed. If you really want to rename, you can always drop 
 the index and recreate it after the rename.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5706) OOM while loading key cache at startup

2013-06-26 Thread Fabien Rousseau (JIRA)

Fabien Rousseau created CASSANDRA-5706:
--

 Summary: OOM while loading key cache at startup
 Key: CASSANDRA-5706
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5706
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Fabien Rousseau
 Attachments: 5706-OOM-while-loading-key-cache-at-startup.patch

Steps to be able to reproduce it :
 - have a heap of 1Gb
 - have a saved key cache without the SSTables

When looking at KeyCacheSerializer.serialize : it always writes a Boolean
When looking at KeyCacheSerializer.deserialize : no Boolean is read if SSTable 
is missing...

In case of a promoted index, RowIndexEntry.serializer.skip(...) should be 
called rather than RowIndexEntry.serializer.skipPromotedIndex(...) (again for 
symmetry between serialization/deserialization)

Attached is a proposed patch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5706) OOM while loading key cache at startup

2013-06-26 Thread Fabien Rousseau (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fabien Rousseau updated CASSANDRA-5706:
---

Attachment: 5706-OOM-while-loading-key-cache-at-startup.patch

 OOM while loading key cache at startup
 --

 Key: CASSANDRA-5706
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5706
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Fabien Rousseau
 Attachments: 5706-OOM-while-loading-key-cache-at-startup.patch


 Steps to be able to reproduce it :
  - have a heap of 1Gb
  - have a saved key cache without the SSTables
 When looking at KeyCacheSerializer.serialize : it always writes a Boolean
 When looking at KeyCacheSerializer.deserialize : no Boolean is read if 
 SSTable is missing...
 In case of a promoted index, RowIndexEntry.serializer.skip(...) should be 
 called rather than RowIndexEntry.serializer.skipPromotedIndex(...) (again for 
 symmetry between serialization/deserialization)
 Attached is a proposed patch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694036#comment-13694036
 ] 

Christian Spriegel commented on CASSANDRA-5704:
---

{quote}Data that was supposed to be gone, could reappear which feels 
semantically different to me.{quote}
I can understand that. My thought was that durable_writes already applies to 
deletes. So I thought why not also apply that pattern to truncate?


I could live with a new test.mode property though (in cassandra.yaml, 
right?). I would even go as far to say that this one should deactivate all 
fsyncs in Cassandra :-)

Nevertheless, for practical reasons I like the idea of using the durable_writes 
flag, because it allows me to use have different behaviour on my dev-laptop 
for...
- ... one keyspace for my junits. These require constant truncation.
- ... my local installation of our application. Here I like a bit more data 
safety, because I not want to set up my testdata all the time.


 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Ensures paging changes to native protocol are properly versioned

Updated Branches:
  refs/heads/trunk 3ec4ff5ed - 8c062d807


Ensures paging changes to native protocol are properly versioned


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8c062d80
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8c062d80
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8c062d80

Branch: refs/heads/trunk
Commit: 8c062d807e4a4c497dbc9a07c5d256ebf92e80fa
Parents: 3ec4ff5
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jun 26 17:46:50 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jun 26 17:46:50 2013 +0200

--
 .../transport/messages/ExecuteMessage.java  | 10 -
 .../transport/messages/QueryMessage.java| 42 +++-
 2 files changed, 32 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c062d80/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java
--
diff --git 
a/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java 
b/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java
index 7c35e42..2d6b67b 100644
--- a/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java
+++ b/src/java/org/apache/cassandra/transport/messages/ExecuteMessage.java
@@ -50,7 +50,10 @@ public class ExecuteMessage extends Message.Request
 values.add(CBUtil.readValue(body));
 
 ConsistencyLevel consistency = CBUtil.readConsistencyLevel(body);
-int resultPageSize = body.readInt();
+
+int resultPageSize = -1;
+if (version = 2)
+resultPageSize = body.readInt();
 return new ExecuteMessage(id, values, consistency, resultPageSize);
 }
 
@@ -71,7 +74,10 @@ public class ExecuteMessage extends Message.Request
 builder.addValue(value);
 
 builder.add(CBUtil.consistencyLevelToCB(msg.consistency));
-builder.add(CBUtil.intToCB(msg.resultPageSize));
+
+assert msg.resultPageSize == -1 || version = 2;
+if (version = 2)
+builder.add(CBUtil.intToCB(msg.resultPageSize));
 return builder.build();
 }
 };

http://git-wip-us.apache.org/repos/asf/cassandra/blob/8c062d80/src/java/org/apache/cassandra/transport/messages/QueryMessage.java
--
diff --git a/src/java/org/apache/cassandra/transport/messages/QueryMessage.java 
b/src/java/org/apache/cassandra/transport/messages/QueryMessage.java
index 860c404..1ec556c 100644
--- a/src/java/org/apache/cassandra/transport/messages/QueryMessage.java
+++ b/src/java/org/apache/cassandra/transport/messages/QueryMessage.java
@@ -45,18 +45,20 @@ public class QueryMessage extends Message.Request
 {
 String query = CBUtil.readLongString(body);
 ConsistencyLevel consistency = CBUtil.readConsistencyLevel(body);
-int resultPageSize = body.readInt();
-ListByteBuffer values;
-if (body.readable())
-{
-int paramCount = body.readUnsignedShort();
-values = paramCount == 0 ? Collections.ByteBufferemptyList() 
: new ArrayListByteBuffer(paramCount);
-for (int i = 0; i  paramCount; i++)
- values.add(CBUtil.readValue(body));
-}
-else
+
+int resultPageSize = -1;
+ListByteBuffer values = Collections.emptyList();
+
+if (version = 2)
 {
-values = Collections.emptyList();
+resultPageSize = body.readInt();
+if (body.readable())
+{
+int paramCount = body.readUnsignedShort();
+values = paramCount == 0 ? 
Collections.ByteBufferemptyList() : new ArrayListByteBuffer(paramCount);
+for (int i = 0; i  paramCount; i++)
+values.add(CBUtil.readValue(body));
+}
 }
 return new QueryMessage(query, values, consistency, 
resultPageSize);
 }
@@ -70,16 +72,20 @@ public class QueryMessage extends Message.Request
 //   - Number of values
 //   - The values
 int vs = msg.values.size();
-CBUtil.BufferBuilder builder = new CBUtil.BufferBuilder(3 + (vs  
0 ? 1 : 0), 0, vs);
+assert (msg.resultPageSize == -1  vs == 0) || version = 2 : 
Version 1 of the protocol support neither a page size nor values;
+
+CBUtil.BufferBuilder builder = new CBUtil.BufferBuilder(2 + 
(version == 1 ? 0 : 1 + (vs  0 ? 1 : 0)), 0, vs);

[jira] [Commented] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694037#comment-13694037
 ] 

Christian Spriegel commented on CASSANDRA-5704:
---

{quote}I'm not sure I'm convinced that we should not save the truncation record 
when durable writes are disabled.{quote}
My understanding was that my patch only stops it from flushing. The record is 
still being written to a memtable, right?


 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5707) Compression chunk_length shouldn't be mandatory


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-5707:


Attachment: 5707.txt

Trivial patch attached.

 Compression chunk_length shouldn't be mandatory
 ---

 Key: CASSANDRA-5707
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5707
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 2.0 beta 1

 Attachments: 5707.txt


 CASSANDRA-5693 introduced a minor regression on trunk in that we refuse 
 compression parameters that don't have the chunk_length option. This is not 
 the case in 1.2 where we just use the default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5707) Compression chunk_length shouldn't be mandatory

Sylvain Lebresne created CASSANDRA-5707:
---

 Summary: Compression chunk_length shouldn't be mandatory
 Key: CASSANDRA-5707
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5707
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 2.0 beta 1
 Attachments: 5707.txt

CASSANDRA-5693 introduced a minor regression on trunk in that we refuse 
compression parameters that don't have the chunk_length option. This is not the 
case in 1.2 where we just use the default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5151) Implement better way of eliminating compaction left overs.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694059#comment-13694059
 ] 

Sylvain Lebresne commented on CASSANDRA-5151:
-

What's the status of this? Is there something more to do?

 Implement better way of eliminating compaction left overs.
 --

 Key: CASSANDRA-5151
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5151
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.1.3
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0 beta 1

 Attachments: 
 0001-move-scheduling-MeteredFlusher-to-CassandraDaemon.patch, 5151-1.2.txt, 
 5151-v2.txt


 This is from discussion in CASSANDRA-5137. Currently we skip loading SSTables 
 that are left over from incomplete compaction to not over-count counter, but 
 the way we track compaction completion is not secure.
 One possible solution is to create system CF like:
 {code}
 create table compaction_log (
   id uuid primary key,
   inputs setint,
   outputs setint
 );
 {code}
 to track incomplete compaction.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5429) Update scrub and scrubtest for single-pass compaction format


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694063#comment-13694063
 ] 

Sylvain Lebresne commented on CASSANDRA-5429:
-

[~jasobrown] ping. Is this still on your radar? Cause it does sound like 
something we need to fix for 2.0.

 Update scrub and scrubtest for single-pass compaction format
 

 Key: CASSANDRA-5429
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5429
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0 beta 1
Reporter: Jonathan Ellis
Assignee: Jason Brown
 Fix For: 2.0 beta 1




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5014) Convert timeout from milli second to micro second.


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694082#comment-13694082
 ] 

Sylvain Lebresne commented on CASSANDRA-5014:
-

As far as 2.0 is concerned, I'd vote to sticking to retry timeouts. For request 
timeouts, it's not that I'm opposed to making them more precise in practice, 
but currently we don't even really respect what we pretend those do (typically, 
read_request_timeout_in_ms is said to be How long the coordinator should wait 
for read operations to complete, but in theory a coordinator may wait a lot 
more than that if there is some repair results to wait on and some short-read 
retry).

Now if we do CASSANDRA-5239 (which I hope we will), it'll change the meaning of 
those timeout to be actual user query timeouts. It felt that this will be a 
good time then to switch the unit for them, but doing it now will just probably 
force upgrading people to change twice their yaml (for 2.0 and then for 2.1) 
for no big value imo.

There's also the fact that I'd really want to get 2.0 beta1 out of the door so 
if we can restrict the scope of this ticket a bit, I like it too. 

 Convert timeout from milli second to micro second.
 --

 Key: CASSANDRA-5014
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5014
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0 beta 1
Reporter: Vijay
Assignee: Vijay
Priority: Trivial
 Fix For: 2.0 beta 1

 Attachments: 0001-CASSANDRA-5014.patch


 Convert all the timeouts to microseconds.
 Jonathan's comment from CASSANDRA-4705
 {quote}
 millis may be too coarse a grain here, especially for Custom settings. 
 Currently an in-memory read will typically be under 2ms and it's quite 
 possible we can get that down to 1 if we can purge some of the latency 
 between stages. Might as well use micros since Timer gives it to us for free
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5014) Convert timeout from milli second to micro second.

2013-06-26 Thread Lior Golan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694091#comment-13694091
 ] 

Lior Golan commented on CASSANDRA-5014:
---

Why not keep the timeouts in millisecond (in the configuration) but make them 
double, and then convert them to microseconds internally. This way you keep 
compatibility with old timeout values while allowing users to set microsecond 
level timeouts if they wish

 Convert timeout from milli second to micro second.
 --

 Key: CASSANDRA-5014
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5014
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0 beta 1
Reporter: Vijay
Assignee: Vijay
Priority: Trivial
 Fix For: 2.0 beta 1

 Attachments: 0001-CASSANDRA-5014.patch


 Convert all the timeouts to microseconds.
 Jonathan's comment from CASSANDRA-4705
 {quote}
 millis may be too coarse a grain here, especially for Custom settings. 
 Currently an in-memory read will typically be under 2ms and it's quite 
 possible we can get that down to 1 if we can purge some of the latency 
 between stages. Might as well use micros since Timer gives it to us for free
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Ninja-fix native proto spec typo

Updated Branches:
  refs/heads/cassandra-1.2 a6ca5d496 - 40112ec95


Ninja-fix native proto spec typo


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/40112ec9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/40112ec9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/40112ec9

Branch: refs/heads/cassandra-1.2
Commit: 40112ec953298de894a57d27895202488b3b5c3d
Parents: a6ca5d4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 26 00:09:41 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 26 00:09:41 2013 +0300

--
 doc/native_protocol.spec | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/40112ec9/doc/native_protocol.spec
--
diff --git a/doc/native_protocol.spec b/doc/native_protocol.spec
index fb709e3..b7a1de5 100644
--- a/doc/native_protocol.spec
+++ b/doc/native_protocol.spec
@@ -273,7 +273,7 @@ Table of Contents
   Prepare a query for later execution (through EXECUTE). The body consists of
   the CQL query to prepare as a [long string].
 
-  The server will respond with a RESULT message with a `prepared` kind 
(0x3,
+  The server will respond with a RESULT message with a `prepared` kind (0x0004,
   see Section 4.2.5).

[jira] [Created] (CASSANDRA-5708) Add DELETE ... IF EXISTS to CQL3

Sylvain Lebresne created CASSANDRA-5708:
---

 Summary: Add DELETE ... IF EXISTS to CQL3
 Key: CASSANDRA-5708
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5708
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0.1


I've been slightly lazy in CASSANDRA-5443 and didn't added a {{DELETE .. IF 
EXISTS}} syntax to CQL because it wasn't immediately clear what was the correct 
condition to use for the IF EXISTS. But at least for CQL3 tables, this is in 
fact pretty easy to do using the row marker so we should probably add it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5702) ALTER RENAME is broken in trunk


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694099#comment-13694099
 ] 

Aleksey Yeschenko commented on CASSANDRA-5702:
--

+1

 ALTER RENAME is broken in trunk
 ---

 Key: CASSANDRA-5702
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5702
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.0 beta 1

 Attachments: 5702.txt


 CASSANDRA-5125 has broken {{ALTER RENAME}} when the column is a default alias 
 (for thrift column families where the PK columns haven't been renamed yet).
 The problem is basically that while we assign default aliases to PK columns 
 when they don't have one, we currently fake those default aliases and do 
 not persist them. Concretely, CFDefinition is aware of them, but CFMetaData 
 is not, which break renaming post CASSANDRA-5125.
 We could fix rename punctually, but there is another related problem: for the 
 same reason, if you try to create an index on a column that is a non-renamed 
 default alias, this doesn't work with the arguably confusing message No 
 column definition found for column X. Here again, we could fix it 
 punctually, but it starts to sound like we need a more general fix.
 So I suggest stopping to fake those default aliases, but instead to just 
 create real aliases (that are known of CFMetaData and persisted in the 
 schema) when there is none. After all, from a user point of view, why should 
 a default column name be any special. And on top of fixing the issues above, 
 this also:
 # fix CASSANDRA-5489 in a somewhat simpler way
 # makes it easier for clients reading the schema CFs. They won't to infer the 
 default aliases anymore.
 The only theoretical downside is that we lose the information that a given 
 CQL3 column name is one assigned by default versus one set up by the user, 
 but given the user can rename those column names anyway, not sure this 
 matters in any way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4206) AssertionError: originally calculated column size of 629444349 but now it is 588008950

2013-06-26 Thread Arya Goudarzi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694104#comment-13694104
 ] 

Arya Goudarzi commented on CASSANDRA-4206:
--

Any clues? We are not able to get one of our critical CFs in good shape as we 
cannot run compactions on it and its performance is deteriorating. 

 AssertionError: originally calculated column size of 629444349 but now it is 
 588008950
 --

 Key: CASSANDRA-4206
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4206
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0.9
 Environment: Debian Squeeze Linux, kernel 2.6.32, sun-java6-bin 
 6.26-0squeeze1
Reporter: Patrik Modesto

 I've 4 node cluster of Cassandra 1.0.9. There is a rfTest3 keyspace with RF=3 
 and one CF with two secondary indexes. I'm importing data into this CF using 
 Hadoop Mapreduce job, each row has less than 10 colkumns. From JMX:
 MaxRowSize:  1597
 MeanRowSize: 369
 And there are some tens of millions of rows.
 It's write-heavy usage and there is a big pressure on each node, there are 
 quite some dropped mutations on each node. After ~12 hours of inserting I see 
 these assertion exceptiona on 3 out of four nodes:
 {noformat}
 ERROR 06:25:40,124 Fatal exception in thread Thread[HintedHandoff:1,1,main]
 java.lang.RuntimeException: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of 629444349 but 
 now it is 588008950
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:388)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:256)
at 
 org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:84)
at 
 org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:437)
at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
 Caused by: java.util.concurrent.ExecutionException:
 java.lang.AssertionError: originally calculated column size of
 629444349 but now it is 588008950
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at 
 org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:384)
... 7 more
 Caused by: java.lang.AssertionError: originally calculated column size
 of 629444349 but now it is 588008950
at 
 org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:124)
at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:160)
at 
 org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:161)
at 
 org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:380)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
 {noformat}
 Few lines regarding Hints from the output.log:
 {noformat}
  INFO 06:21:26,202 Compacting large row 
 system/HintsColumnFamily:7000 (1712834057 bytes) 
 incrementally
  INFO 06:22:52,610 Compacting large row 
 system/HintsColumnFamily:1000 (2616073981 bytes) 
 incrementally
  INFO 06:22:59,111 flushing high-traffic column family CFS(Keyspace='system', 
 ColumnFamily='HintsColumnFamily') (estimated 305147360 bytes)
  INFO 06:22:59,813 Enqueuing flush of 
 Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, 
 7452 ops)
  INFO 06:22:59,814 Writing 
 Memtable-HintsColumnFamily@833933926(3814342/305147360 serialized/live bytes, 
 7452 ops)
 {noformat}
 I think the problem may be somehow connected to an IntegerType secondary 
 index. I had a different problem with CF with two secondary indexes, the 
 first UTF8Type, the second IntegerType. After a few hours of inserting data 
 in the afternoon and midnight repair+compact, the next day I couldn't find 
 any row using the IntegerType secondary index. The output was like this:
 {noformat}
 [default@rfTest3] get IndexTest where col1 = 
 '3230727:http://zaskolak.cz/download.php';
 ---
 RowKey: 3230727:8383582:http://zaskolak.cz/download.php
 = (column=col1, value=3230727:http://zaskolak.cz/download.php,

git commit: Fix ALTER RENAME post-5125

Updated Branches:
  refs/heads/trunk 8c062d807 - 67435b528


Fix ALTER RENAME post-5125

patch by slebresne; reviewed by iamaleksey for CASSANDRA-5702


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/67435b52
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/67435b52
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/67435b52

Branch: refs/heads/trunk
Commit: 67435b528dd474bd25fc90eaace6e6786f75ce04
Parents: 8c062d8
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jun 26 19:34:27 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jun 26 19:34:27 2013 +0200

--
 CHANGES.txt |   1 +
 .../org/apache/cassandra/config/CFMetaData.java | 237 ---
 .../cassandra/config/ColumnDefinition.java  |  10 +-
 .../org/apache/cassandra/config/KSMetaData.java |   1 +
 .../cassandra/cql/AlterTableStatement.java  |   2 +-
 .../apache/cassandra/cql/QueryProcessor.java|  29 +--
 .../org/apache/cassandra/cql/WhereClause.java   |  11 +-
 .../org/apache/cassandra/cql3/CFDefinition.java |  33 +--
 .../cql3/statements/AlterTableStatement.java|  26 --
 src/java/org/apache/cassandra/db/DefsTable.java |   6 +
 .../apache/cassandra/config/CFMetaDataTest.java |  28 ++-
 .../org/apache/cassandra/config/DefsTest.java   |   2 +-
 12 files changed, 191 insertions(+), 195 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/67435b52/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index c0bafb3..4973987 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -64,6 +64,7 @@
  * Conditional create/drop ks/table/index statements in CQL3 (CASSANDRA-2737)
  * more pre-table creation property validation (CASSANDRA-5693)
  * Redesign repair messages (CASSANDRA-5426)
+ * Fix ALTER RENAME post-5125 (CASSANDRA-5702)
 
 
 1.2.7

http://git-wip-us.apache.org/repos/asf/cassandra/blob/67435b52/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 15713bb..43d7297 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -22,6 +22,7 @@ import java.lang.reflect.Constructor;
 import java.lang.reflect.InvocationTargetException;
 import java.lang.reflect.Method;
 import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
 import java.util.*;
 import java.util.Map.Entry;
 
@@ -80,7 +81,6 @@ public final class CFMetaData
 public final static int DEFAULT_MIN_COMPACTION_THRESHOLD = 4;
 public final static int DEFAULT_MAX_COMPACTION_THRESHOLD = 32;
 public final static Class? extends AbstractCompactionStrategy 
DEFAULT_COMPACTION_STRATEGY_CLASS = SizeTieredCompactionStrategy.class;
-public final static ByteBuffer DEFAULT_KEY_NAME = 
ByteBufferUtil.bytes(KEY);
 public final static Caching DEFAULT_CACHING_STRATEGY = Caching.KEYS_ONLY;
 public final static int DEFAULT_DEFAULT_TIME_TO_LIVE = 0;
 public final static SpeculativeRetry DEFAULT_SPECULATIVE_RETRY = new 
SpeculativeRetry(SpeculativeRetry.RetryType.NONE, 0);
@@ -381,6 +381,10 @@ public final class CFMetaData
  * clustering key ones, those list are ordered by the component index of 
the
  * elements.
  */
+public static final String DEFAULT_KEY_ALIAS = key;
+public static final String DEFAULT_COLUMN_ALIAS = column;
+public static final String DEFAULT_VALUE_ALIAS = value;
+
 private volatile MapByteBuffer, ColumnDefinition column_metadata = new 
HashMapByteBuffer,ColumnDefinition();
 private volatile ListColumnDefinition partitionKeyColumns;  // Always of 
size keyValidator.componentsCount, null padded if necessary
 private volatile ListColumnDefinition clusteringKeyColumns; // Of size 
comparator.componentsCount or comparator.componentsCount -1, null padded if 
necessary
@@ -402,11 +406,11 @@ public final class CFMetaData
 public CFMetaData dcLocalReadRepairChance(double prop) 
{dcLocalReadRepairChance = prop; return this;}
 public CFMetaData replicateOnWrite(boolean prop) {replicateOnWrite = prop; 
return this;}
 public CFMetaData gcGraceSeconds(int prop) {gcGraceSeconds = prop; return 
this;}
-public CFMetaData defaultValidator(AbstractType? prop) {defaultValidator 
= prop; updateCfDef(); return this;}
-public CFMetaData keyValidator(AbstractType? prop) {keyValidator = prop; 
updateCfDef(); return this;}
+public CFMetaData defaultValidator(AbstractType? prop) {defaultValidator 
= prop; return this;}
+public CFMetaData

[1/2] git commit: cqlsh: Add ALTER TABLE RENAME autocompletion and docs

Updated Branches:
  refs/heads/trunk 67435b528 - 6258c58cd


cqlsh: Add ALTER TABLE RENAME autocompletion and docs


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b97b490d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b97b490d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b97b490d

Branch: refs/heads/trunk
Commit: b97b490dffda91284befa9049255ae477ea8114c
Parents: 40112ec
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 26 20:41:38 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 26 20:41:38 2013 +0300

--
 pylib/cqlshlib/cql3handling.py |  2 ++
 pylib/cqlshlib/helptopics.py   | 33 +++--
 2 files changed, 33 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b97b490d/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 958b9ae..0e631ea 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1239,6 +1239,8 @@ syntax_rules += r'''
   | ADD newcol=cident storageType
   | DROP existcol=cident
   | WITH cfamProperty ( AND cfamProperty )*
+  | RENAME existcol=cident TO newcol=cident
+ ( AND existcol=cident TO newcol=cident )*
   ;
 '''
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b97b490d/pylib/cqlshlib/helptopics.py
--
diff --git a/pylib/cqlshlib/helptopics.py b/pylib/cqlshlib/helptopics.py
index c9dae31..cdbb8f4 100644
--- a/pylib/cqlshlib/helptopics.py
+++ b/pylib/cqlshlib/helptopics.py
@@ -902,8 +902,37 @@ class CQL3HelpTopics(CQLHelpTopics):
 print   HELP CREATE_USER;
 
 def help_alter(self):
-super(CQL3HelpTopics, self).help_alter()
-print   HELP ALTER_USER;
+print 
+ALTER TABLE tablename ALTER columnname TYPE type;
+ALTER TABLE tablename ADD columnname type;
+ALTER TABLE tablename RENAME columnname TO columnname
+[AND columnname TO columnname]
+ALTER TABLE tablename WITH optionname = val [AND optionname = 
val [...]];
+
+An ALTER statement is used to manipulate table metadata. It allows you
+to add new typed columns, drop existing columns, change the data
+storage type of existing columns, or change table properties.
+No results are returned.
+
+See one of the following for more information:
+
+  HELP ALTER_ALTER;
+  HELP ALTER_ADD;
+  HELP ALTER_DROP;
+  HELP ALTER_RENAME;
+  HELP ALTER_WITH;
+
+
+def help_alter_rename(self):
+print 
+ALTER TABLE: renaming a column
+
+  ALTER TABLE tablename RENAME columnname TO columnname
+  [AND columnname TO columnname]
+
+The ALTER TABLE ... RENAME variant renames a typed column in a column
+family.
+
 
 def help_drop(self):
 super(CQL3HelpTopics, self).help_drop()

[2/2] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk

Conflicts:
pylib/cqlshlib/helptopics.py


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6258c58c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6258c58c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6258c58c

Branch: refs/heads/trunk
Commit: 6258c58cdbeae8b69dfe990af22506ed609dcb9a
Parents: 67435b5 b97b490
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 26 20:50:22 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 26 20:50:22 2013 +0300

--
 NEWS.txt   | 14 --
 pylib/cqlshlib/cql3handling.py |  2 ++
 pylib/cqlshlib/helptopics.py   | 53 ++---
 3 files changed, 38 insertions(+), 31 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6258c58c/NEWS.txt
--
diff --cc NEWS.txt
index a0711c7,cb40981..5e784d1
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -8,73 -8,11 +8,69 @@@ upgrade, just in case you need to roll 
  (Cassandra version X + 1 will always be able to read data files created
  by version X, but the inverse is not necessarily the case.)
  
-  HEAD
+ When upgrading major versions of Cassandra, you will be unable to
+ restore snapshots created with the previous major version using the
+ 'sstableloader' tool. You can upgrade the file format of your snapshots
+ using the provided 'sstableupgrade' tool.
+ 
 +2.0.0
 +=
 +
 +Upgrading
 +-
 +- CAS and new features in CQL such as DROP COLUMN assume that cell
 +  timestamps are microseconds-since-epoch.  Do not use these
 +  features if you are using client-specified timestamps with some
 +  other source.
 +- Upgrading is ONLY supported from Cassandra 1.2.5 or later.  This
 +  goes for sstable compatibility as well as network.  When
 +  upgrading from an earlier release, upgrade to 1.2.5 first and
 +  run upgradesstables before proceeding to 2.0.
 +- Replication and strategy options do not accept unknown options anymore.
 +  This was already the case for CQL3 in 1.2 but this is now the case for
 +  thrift too.
 +- auto_bootstrap of a single-token node with no initial_token will
 +  now pick a random token instead of bisecting an existing token
 +  range.  We recommend upgrading to vnodes; failing that, we
 +  recommend specifying initial_token.
 +- reduce_cache_sizes_at, reduce_cache_capacity_to, and
 +  flush_largest_memtables_at options have been removed from 
cassandra.yaml.
 +- CacheServiceMBean.reduceCacheSizes() has been removed.
 +  Use CacheServiceMBean.set{Key,Row}CacheCapacityInMB() instead.
 +- authority option in cassandra.yaml has been deprecated since 1.2.0,
 +  but it has been completely removed in 2.0. Please use 'authorizer' 
option.
 +- ASSUME command has been removed from cqlsh. Use CQL3 blobAsType() and
 +  typeAsBlob() conversion functions instead.
 +  See https://cassandra.apache.org/doc/cql3/CQL.html#blobFun for details.
 +- Inputing blobs as string constants is now fully deprecated in
 +  favor of blob constants. Make sure to update your applications to use
 +  the new syntax while you are still on 1.2 (which supports both string
 +  and blob constants for blob input) before upgrading to 2.0.
 +
 +Operations
 +--
 +- Major compactions, cleanup, scrub, and upgradesstables will interrupt 
 +  any in-progress compactions (but not repair validations) when invoked.
 +- Disabling autocompactions by setting min/max compaction threshold to 0
 +  has been deprecated, instead, use the nodetool commands 
'disableautocompaction'
 +  and 'enableautocompaction' or set the compaction strategy option 
enabled = false
 +- ALTER TABLE DROP has been reenabled for CQL3 tables and has new 
semantics now.
 +  See https://cassandra.apache.org/doc/cql3/CQL.html#alterTableStmt and
 +  https://issues.apache.org/jira/browse/CASSANDRA-3919 for details.
 +- CAS uses gc_grace_seconds to determine how long to keep unused paxos
 +  state around for, or a minimum of three hours.
 +
 +Features
 +
 +- Alias support has been added to CQL3 SELECT statement. Refer to
 +  CQL3 documentation (http://cassandra.apache.org/doc/cql3/CQL.html) for 
details.
 +- JEMalloc support (see memory_allocator in cassandra.yaml)
 +- Experimental triggers support.  See examples/ for how to use.  
Experimental
 +  means tied closely to internal data structures; we plan to decouple 
this in
 +  the future, which will probably break triggers written against this 
initial
 +  API.
 +
 +
- ||| merged common ancestors
- ===
-

git commit: cqlsh: Add ALTER TABLE RENAME autocompletion and docs

Updated Branches:
  refs/heads/cassandra-1.2 40112ec95 - b97b490df


cqlsh: Add ALTER TABLE RENAME autocompletion and docs


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b97b490d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b97b490d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b97b490d

Branch: refs/heads/cassandra-1.2
Commit: b97b490dffda91284befa9049255ae477ea8114c
Parents: 40112ec
Author: Aleksey Yeschenko alek...@apache.org
Authored: Wed Jun 26 20:41:38 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Wed Jun 26 20:41:38 2013 +0300

--
 pylib/cqlshlib/cql3handling.py |  2 ++
 pylib/cqlshlib/helptopics.py   | 33 +++--
 2 files changed, 33 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b97b490d/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 958b9ae..0e631ea 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -1239,6 +1239,8 @@ syntax_rules += r'''
   | ADD newcol=cident storageType
   | DROP existcol=cident
   | WITH cfamProperty ( AND cfamProperty )*
+  | RENAME existcol=cident TO newcol=cident
+ ( AND existcol=cident TO newcol=cident )*
   ;
 '''
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b97b490d/pylib/cqlshlib/helptopics.py
--
diff --git a/pylib/cqlshlib/helptopics.py b/pylib/cqlshlib/helptopics.py
index c9dae31..cdbb8f4 100644
--- a/pylib/cqlshlib/helptopics.py
+++ b/pylib/cqlshlib/helptopics.py
@@ -902,8 +902,37 @@ class CQL3HelpTopics(CQLHelpTopics):
 print   HELP CREATE_USER;
 
 def help_alter(self):
-super(CQL3HelpTopics, self).help_alter()
-print   HELP ALTER_USER;
+print 
+ALTER TABLE tablename ALTER columnname TYPE type;
+ALTER TABLE tablename ADD columnname type;
+ALTER TABLE tablename RENAME columnname TO columnname
+[AND columnname TO columnname]
+ALTER TABLE tablename WITH optionname = val [AND optionname = 
val [...]];
+
+An ALTER statement is used to manipulate table metadata. It allows you
+to add new typed columns, drop existing columns, change the data
+storage type of existing columns, or change table properties.
+No results are returned.
+
+See one of the following for more information:
+
+  HELP ALTER_ALTER;
+  HELP ALTER_ADD;
+  HELP ALTER_DROP;
+  HELP ALTER_RENAME;
+  HELP ALTER_WITH;
+
+
+def help_alter_rename(self):
+print 
+ALTER TABLE: renaming a column
+
+  ALTER TABLE tablename RENAME columnname TO columnname
+  [AND columnname TO columnname]
+
+The ALTER TABLE ... RENAME variant renames a typed column in a column
+family.
+
 
 def help_drop(self):
 super(CQL3HelpTopics, self).help_drop()

[jira] [Commented] (CASSANDRA-5534) Writing wide row causes high CPU usage after compaction


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694121#comment-13694121
 ] 

Sylvain Lebresne commented on CASSANDRA-5534:
-

Would you mind testing that again? I just tried the same test with current 
trunk and it's fast ( 2s for the whole stress).

 Writing wide row causes high CPU usage after compaction
 ---

 Key: CASSANDRA-5534
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5534
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0 beta 1
Reporter: Ryan McGuire
Assignee: Sylvain Lebresne
 Fix For: 2.0

 Attachments: wide_row_stress.trunk.log.txt.gz


 Introduced in commit -e74c13ff08663d306dcc5cdc99c07e9e6c12ca21- (see below) 
 there is a significant slow down when creating a wide row with 
 cassandra-stress:
 Testing with the prior (good) commit I used this to write a single wide row, 
 which completed rather quickly:
 {code}
 $ ccm create -v git:60f09f0121e0801851b9ab017eddf7e326fa05fb wide-row
 Fetching Cassandra updates...
 Cloning Cassandra (from local cache)
 Checking out requested branch (60f09f0121e0801851b9ab017eddf7e326fa05fb)
 Compiling Cassandra 60f09f0121e0801851b9ab017eddf7e326fa05fb ...
 Current cluster is now: wide-row
 $ ccm populate -n 1
 $ ccm start
 $ time ccm node1 stress -c 1 -S 1000 -n 1
 Created keyspaces. Sleeping 1s for propagation.
 total,interval_op_rate,interval_key_rate,latency/95th/99th,elapsed_time
 1,0,0,273.3,273.3,273.3,0
 END
 real  0m7.106s
 user  0m1.710s
 sys   0m0.120s
 {code}
 Using the bugged commit (e74c13ff08663d306dcc5cdc99c07e9e6c12ca21) I get a 
 significant slow down:
 {code}
 02:42 PM:~$ ccm create -v git:e74c13ff08663d306dcc5cdc99c07e9e6c12ca21 
 wide-row
 Fetching Cassandra updates...
 Current cluster is now: wide-row
 02:42 PM:~$ ccm populate -n 1
 02:42 PM:~$ ccm start
 02:42 PM:~$ time ccm node1 stress -c 1 -S 1000 -n 1
 Created keyspaces. Sleeping 1s for propagation.
 total,interval_op_rate,interval_key_rate,latency,95th,99th,elapsed_time
 1,0,0,423.2,423.2,423.2,0
 Total operation time  : 00:00:00
 END
 real  4m16.394s
 user  0m2.230s
 sys   0m0.137s
 {code}
 Interestingly, the commit in question just says it's a merge from 
 cassandra-1.2, but I do not see this same slowdown using that branch, this 
 only occurs in trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Disallow renaming a 2ndary indexed column

Updated Branches:
  refs/heads/trunk 6258c58cd - 29232a88d


Disallow renaming a 2ndary indexed column

patch by slebresne; reviewed by iamaleksey for CASSANDRA-5705


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29232a88
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29232a88
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29232a88

Branch: refs/heads/trunk
Commit: 29232a88d174d331a982e5abc01b0d441ed1c248
Parents: 6258c58
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Wed Jun 26 19:59:47 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Wed Jun 26 20:00:41 2013 +0200

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/config/CFMetaData.java | 6 ++
 2 files changed, 7 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/29232a88/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4973987..a79d2db 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -65,6 +65,7 @@
  * more pre-table creation property validation (CASSANDRA-5693)
  * Redesign repair messages (CASSANDRA-5426)
  * Fix ALTER RENAME post-5125 (CASSANDRA-5702)
+ * Disallow renaming a 2ndary indexed column (CASSANDRA-5705)
 
 
 1.2.7

http://git-wip-us.apache.org/repos/asf/cassandra/blob/29232a88/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 43d7297..6594d75 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -1794,7 +1794,13 @@ public final class CFMetaData
 throw new InvalidRequestException(String.format(Cannot rename 
column %s to %s in table %s; another column of that name already exist, 
strFrom, strTo, cfName));
 
 if (def.type == ColumnDefinition.Type.REGULAR)
+{
 throw new InvalidRequestException(String.format(Cannot rename non 
PRIMARY KEY part %s, strFrom));
+}
+else if (def.isIndexed())
+{
+throw new InvalidRequestException(String.format(Cannot rename 
column %s because it is secondary indexed, strFrom));
+}
 
 ColumnDefinition newDef = def.cloneWithNewName(to);
 // don't call addColumnDefinition/removeColumnDefition because we want 
to avoid recomputing

[jira] [Commented] (CASSANDRA-5707) Compression chunk_length shouldn't be mandatory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694128#comment-13694128
 ] 

Aleksey Yeschenko commented on CASSANDRA-5707:
--

+1

 Compression chunk_length shouldn't be mandatory
 ---

 Key: CASSANDRA-5707
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5707
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Trivial
 Fix For: 2.0 beta 1

 Attachments: 5707.txt


 CASSANDRA-5693 introduced a minor regression on trunk in that we refuse 
 compression parameters that don't have the chunk_length option. This is not 
 the case in 1.2 where we just use the default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694164#comment-13694164
 ] 

Pavel Yaskevich commented on CASSANDRA-5661:


bq. While in theory I like the idea of the searchers in most cases expiring 
quicker with timers, in practice since these are on the heap, and LCS defaults 
to only 5 MB files per levels, you can have A LOT of sstables, 200 per GB... 
Which as Jonathan Ellis mentioned makes # of sstables * concurrent readers a 
big problem, which was the problem hit above. So we really need to bound memory 
usage with a hard cap, not the fuzzy cap of how many can I open in X seconds

When I was talking about concurrent readers I actually meant the max number of 
threads in READ stage (concurrent_reads in yaml, default 32) that could be 
running in the same time, that is the worst case limit of cache queue per 
SSTable.
 
I'm not convinced that LRU on SSTable would work better then [C]RAR instance 
expiration for our use case because due to replacement mechanism it creates 
jitter in latencies grater than p75 once global threshold is reached, which in 
production systems create situation when people can't explain why latencies 
suddenly degraded without increasing amount of data.

I'm not sure that I follow what you mean by last part, we are not trying to 
limit number of open for period of time but rather expire items that been 
returned to the queue after time period. I agree that we can add upper limit on 
memory usage just like we did for key/row caches, which in combination with 
expiring would yield predictable behavior.

bq. 1k reqs/sec hitting a 4 level deep LCS CF could mean 4k Reader's (~500 MB) 
created per second. Now that I say that out loud, I almost think we should go 
back to not caching these at all so they always just get recycled in young gen 
and never have a chance to hit old gen. I guess it could be a config parameter 
which defaults on for STCS and off for LCS. Since STCS work loads have no where 
near as many sstables.

It sounds like memory overhead caused by compression for your use-case is too 
big with or without caching (it's around 79KB = 64KB default chunk + snappy 
overhead) per file, even if you drop caching your allocation rate would cause 
[C]RAR buffers to be promoted and cause fragmentation in the old gen which 
could cause long remark phrase of CMS (can be battled with 
CMSScavengeBeforeRemark option) and as CMS is not compacting you can suffer 
from FullGC more frequently. That is what we are trying to battle with caching 
[C]RAR instances (and their buffers) as if we allocate those close in time and 
reuse, it's more luckily that they are going to be laid out closely plus 
removes portion of work from young gen collector.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-5696:


Attachment: 5696.txt

There are a few different ways this can be solved, but it's obvious that the 
most correct thing to do is correct the order in VV.  This too turns out to be 
workable though, as long as you either a) do a full ring restart (to wipe the 
dead states) or b) do not upgrade within 72 hours of decommissioning a node.  
Almost everyone will likely fall into group b) but if they don't, and don't 
read the warning in NEWS.txt the possible things that can happen are:

* they receive a harmless NumberFormatException everywhere
* they carry a useless dead state around for a token that doesn't exist for an 
indeterminate amount of time (which assassinate could fix)
* the timestamp used for the decom expiration matches a node's token and it 
will need to be  shutdown and rebootstrapped

None of these seem world-ending, with the last one being the worst and also 
least likely to occur, so here's a patch where we just change the VV order and 
update NEWS.

 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-5696:


Reviewer: jasobrown
  Tester: enigmacurry

 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5661) Discard pooled readers for cold data

2013-06-26 Thread Jeremiah Jordan (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremiah Jordan updated CASSANDRA-5661:
---

Attachment: DominatorTree.png
Histogram.png

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7

 Attachments: DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

2013-06-26 Thread Jeremiah Jordan (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694202#comment-13694202
]

Jeremiah Jordan commented on CASSANDRA-5661:

Screen shots attached to show the issue.
Histogram.png show my histogram. Over 2 GB is being retained by CRAR objects
on a node with only 80 GB of data, most of that in LCS CF's with 20 MB sstable
size, and some ine LCS CF's with 10 MB sstable size.
!Histogram.png!

Here is the dominator tree for that same heap dump.
!DominatorTree.png!

The problem is that most of those sstables have 1 MB or more of CRAR buffers
sitting there waiting to be used. If we do a timer with an absolute cap, even
better, but we need some kind of cap.

The compression overhead itself isn't really an issue, the issue is the cached
CRAR objects/buffers.

Discard pooled readers for cold data

Attachments: DominatorTree.png, Histogram.png

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694208#comment-13694208
 ] 

Deepak Rosario Pancras commented on CASSANDRA-5234:
---

@Alex I pulled the branch CASSANDRA-5234 and tried building it. But the build 
fails when compiling.

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694214#comment-13694214
 ] 

Alex Liu commented on CASSANDRA-5234:
-

It works for me.

try the commands

{code}
git checkout cassandra-1.2
patch -p1  5234-3-1.2branch.txt 
{code}

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5675) cqlsh shouldn't display null for empty values


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694218#comment-13694218
 ] 

Brandon Williams commented on CASSANDRA-5675:
-

+1

 cqlsh shouldn't display null for empty values
 ---

 Key: CASSANDRA-5675
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5675
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Aleksey Yeschenko
Priority: Minor
  Labels: cqlsh
 Fix For: 1.2.7

 Attachments: 5675.txt


 For historical reason (and compatibility with thrift), all type support an 
 empty value, even type like int for which it doesn't really make sense (see 
 CASSANDRA-5674 too on that subject).
 If you input such an empty value for a type like int, cqlsh will display it 
 as null:
 {noformat}
 cqlsh:ks CREATE TABLE test (k text PRIMARY KEY, v int);
 cqlsh:ks INSERT INTO test(k, v) VALUES ('someKey', blobAsInt(0x));
 cqlsh:ks SELECT * FROM test;
  k   | v
 -+--
  someKey | null
 {noformat} 
 But that's not correct, it suggests {{v}} has no value but that's not true, 
 it has a value, it's just an empty one.
 Now, one may argue support empty values for a type like int is broken, and I 
 would agree with that. But thrift allows it so we probably need to preserve 
 that behavior for compatibility sake. And I guess the need to use blobAsInt 
 at least make it clear that it's kind of a hack.
 That being said, cqlsh should not display null as this is confusing. Instead 
 I'd suggest either displaying nothing (that's how an empty string is 
 displayed after all), or to just go with some random explicit syntax like say 
 [empty value]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694225#comment-13694225
 ] 

Jason Brown commented on CASSANDRA-5696:


After we make this change, won't the same rules apply to 1.2.{0..6} - 
1.2.7+/2.0 upgrades? The VV fields will again be inverted, although arguably in 
a more internally consistent manner (with 1.1)?


 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694225#comment-13694225
 ] 

Jason Brown edited comment on CASSANDRA-5696 at 6/26/13 8:38 PM:
-

After we make this change, won't the same rules apply to 1.2.\{0..6\} - 
1.2.7+/2.0 upgrades? The VV fields will again be inverted, although arguably in 
a more internally consistent manner (with 1.1)?


  was (Author: jasobrown):
After we make this change, won't the same rules apply to 1.2.{0..6} - 
1.2.7+/2.0 upgrades? The VV fields will again be inverted, although arguably in 
a more internally consistent manner (with 1.1)?

  
 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694225#comment-13694225
 ] 

Jason Brown edited comment on CASSANDRA-5696 at 6/26/13 8:38 PM:
-

After we make this change, won't the same problems apply to 1.2.\{0..6\} - 
1.2.7+/2.0 upgrades? The VV fields will again be inverted, although arguably in 
a more internally consistent manner (with 1.1)?


  was (Author: jasobrown):
After we make this change, won't the same rules apply to 1.2.\{0..6\} - 
1.2.7+/2.0 upgrades? The VV fields will again be inverted, although arguably in 
a more internally consistent manner (with 1.1)?

  
 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5696) Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694228#comment-13694228
 ] 

Jason Brown commented on CASSANDRA-5696:


Thought a little bit more, and I think this patch is fine. +1

 Upgrading to cassandra-1.2 with a dead LEFT state from 1.1 causes problems
 --

 Key: CASSANDRA-5696
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5696
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.0
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 1.2.7

 Attachments: 5696.txt


 In 1.1, we wrote LEFT states as LEFT,token,expiretime in gossip.  However in 
 1.2, VersionValue serializes this to LEFT,expiretime,tokens and causes the 
 upgrade 1.2 to try and parse it this way as well, causing it to try to parse 
 the token as an expiretime.
 Another wrinkle to this is assassinate still writes it the old way: 
 LEFT,tokens,expiretime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk

Conflicts:
CHANGES.txt
NEWS.txt
src/java/org/apache/cassandra/gms/VersionedValue.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1dacd2b6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1dacd2b6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1dacd2b6

Branch: refs/heads/trunk
Commit: 1dacd2b6d1e76a4a55181e7a78978f3a9335fd32
Parents: 29232a8 54266ea
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 15:56:59 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 15:56:59 2013 -0500

--
 CHANGES.txt   |  2 +-
 NEWS.txt  | 10 ++
 src/java/org/apache/cassandra/gms/VersionedValue.java |  4 ++--
 3 files changed, 13 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1dacd2b6/CHANGES.txt
--
diff --cc CHANGES.txt
index a79d2db,cb3fede..1c5abd0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,77 -1,5 +1,77 @@@
 +2.0
 + * Removed on-heap row cache (CASSANDRA-5348)
 + * use nanotime consistently for node-local timeouts (CASSANDRA-5581)
 + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577)
 + * Experimental triggers (CASSANDRA-1311)
 + * JEMalloc support for off-heap allocation (CASSANDRA-3997)
 + * Single-pass compaction (CASSANDRA-4180)
 + * Removed token range bisection (CASSANDRA-5518)
 + * Removed compatibility with pre-1.2.5 sstables and network messages
 +   (CASSANDRA-5511)
 + * removed PBSPredictor (CASSANDRA-5455)
 + * CAS support (CASSANDRA-5062, 5441, 5442, 5443, 5619)
 + * Leveled compaction performs size-tiered compactions in L0 
 +   (CASSANDRA-5371, 5439)
 + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339)
 + * Log when a node is down longer than the hint window (CASSANDRA-4554)
 + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917)
 + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407)
 + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430)
 + * Change Message IDs to ints (CASSANDRA-5307)
 + * Move sstable level information into the Stats component, removing the
 +   need for a separate Manifest file (CASSANDRA-4872)
 + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199)
 + * make index_interval configurable per columnfamily (CASSANDRA-3961, 
CASSANDRA-5650)
 + * add default_time_to_live (CASSANDRA-3974)
 + * add memtable_flush_period_in_ms (CASSANDRA-4237)
 + * replace supercolumns internally by composites (CASSANDRA-3237, 5123)
 + * upgrade thrift to 0.9.0 (CASSANDRA-3719)
 + * drop unnecessary keyspace parameter from user-defined compaction API 
 +   (CASSANDRA-5139)
 + * more robust solution to incomplete compactions + counters (CASSANDRA-5151)
 + * Change order of directory searching for c*.in.sh (CASSANDRA-3983)
 + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271)
 + * Allow custom configuration loader (CASSANDRA-5045)
 + * Remove memory emergency pressure valve logic (CASSANDRA-3534)
 + * Reduce request latency with eager retry (CASSANDRA-4705)
 + * cqlsh: Remove ASSUME command (CASSANDRA-5331)
 + * Rebuild BF when loading sstables if bloom_filter_fp_chance
 +   has changed since compaction (CASSANDRA-5015)
 + * remove row-level bloom filters (CASSANDRA-4885)
 + * Change Kernel Page Cache skipping into row preheating (disabled by default)
 +   (CASSANDRA-4937)
 + * Improve repair by deciding on a gcBefore before sending
 +   out TreeRequests (CASSANDRA-4932)
 + * Add an official way to disable compactions (CASSANDRA-5074)
 + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919)
 + * Add binary protocol versioning (CASSANDRA-5436)
 + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530)
 + * Add alias support to SELECT statement (CASSANDRA-5075)
 + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541)
 + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579)
 + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585)
 + * Track max/min column names in sstables to be able to optimize slice
 +   queries (CASSANDRA-5514, CASSANDRA-5595, CASSANDRA-5600)
 + * Binary protocol: allow batching already prepared statements 
(CASSANDRA-4693)
 + * Allow preparing timestamp, ttl and limit in CQL3 queries (CASSANDRA-4450)
 + * Support native link w/o JNA in Java7 (CASSANDRA-3734)
 + * Use SASL authentication in binary protocol v2 (CASSANDRA-5545)
 + * Replace Thrift HsHa with LMAX Disruptor based implementation 
(CASSANDRA-5582)
 + * cqlsh: Add row count to SELECT output

[2/3] git commit: Fix backward VV serialization of the LEFT state

Fix backward VV serialization of the LEFT state


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54266ea7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54266ea7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54266ea7

Branch: refs/heads/trunk
Commit: 54266ea705b20e12a6f03dab69fa5d6b05e160a7
Parents: b97b490
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 15:52:43 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 15:52:43 2013 -0500

--
 CHANGES.txt   |  3 +++
 NEWS.txt  | 10 ++
 src/java/org/apache/cassandra/gms/VersionedValue.java |  4 ++--
 3 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2931916..cb3fede 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+1.2.7
+ * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
+
 1.2.6
  * Fix tracing when operation completes before all responses arrive 
(CASSANDRA-5668)
  * Fix cross-DC mutation forwarding (CASSANDRA-5632)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index cb40981..d4d127b 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,16 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+1.2.7
+=
+Upgrading
+-
+- If you have decommissioned a node in the past 72 hours, it is imperative
+  that you not upgrade until such time has passed, or do a full cluster
+  restart (not rolling) before beginning the upgrade.  This only applies to
+  decommission, not removetoken.
+
+
 1.2.6
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/src/java/org/apache/cassandra/gms/VersionedValue.java
--
diff --git a/src/java/org/apache/cassandra/gms/VersionedValue.java 
b/src/java/org/apache/cassandra/gms/VersionedValue.java
index 5841c4b..60459c8 100644
--- a/src/java/org/apache/cassandra/gms/VersionedValue.java
+++ b/src/java/org/apache/cassandra/gms/VersionedValue.java
@@ -148,8 +148,8 @@ public class VersionedValue implements 
ComparableVersionedValue
 public VersionedValue left(CollectionToken tokens, long expireTime)
 {
 return new VersionedValue(versionString(VersionedValue.STATUS_LEFT,
-Long.toString(expireTime),
-makeTokenString(tokens)));
+makeTokenString(tokens),
+Long.toString(expireTime)));
 }
 
 public VersionedValue moving(Token token)

[1/3] git commit: Fix backward VV serialization of the LEFT state

Updated Branches:
  refs/heads/cassandra-1.2 b97b490df - 54266ea70
  refs/heads/trunk 29232a88d - 1dacd2b6d


Fix backward VV serialization of the LEFT state


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54266ea7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54266ea7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54266ea7

Branch: refs/heads/cassandra-1.2
Commit: 54266ea705b20e12a6f03dab69fa5d6b05e160a7
Parents: b97b490
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 15:52:43 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 15:52:43 2013 -0500

--
 CHANGES.txt   |  3 +++
 NEWS.txt  | 10 ++
 src/java/org/apache/cassandra/gms/VersionedValue.java |  4 ++--
 3 files changed, 15 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2931916..cb3fede 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+1.2.7
+ * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
+
 1.2.6
  * Fix tracing when operation completes before all responses arrive 
(CASSANDRA-5668)
  * Fix cross-DC mutation forwarding (CASSANDRA-5632)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index cb40981..d4d127b 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,16 @@ restore snapshots created with the previous major version 
using the
 'sstableloader' tool. You can upgrade the file format of your snapshots
 using the provided 'sstableupgrade' tool.
 
+1.2.7
+=
+Upgrading
+-
+- If you have decommissioned a node in the past 72 hours, it is imperative
+  that you not upgrade until such time has passed, or do a full cluster
+  restart (not rolling) before beginning the upgrade.  This only applies to
+  decommission, not removetoken.
+
+
 1.2.6
 =
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/54266ea7/src/java/org/apache/cassandra/gms/VersionedValue.java
--
diff --git a/src/java/org/apache/cassandra/gms/VersionedValue.java 
b/src/java/org/apache/cassandra/gms/VersionedValue.java
index 5841c4b..60459c8 100644
--- a/src/java/org/apache/cassandra/gms/VersionedValue.java
+++ b/src/java/org/apache/cassandra/gms/VersionedValue.java
@@ -148,8 +148,8 @@ public class VersionedValue implements 
ComparableVersionedValue
 public VersionedValue left(CollectionToken tokens, long expireTime)
 {
 return new VersionedValue(versionString(VersionedValue.STATUS_LEFT,
-Long.toString(expireTime),
-makeTokenString(tokens)));
+makeTokenString(tokens),
+Long.toString(expireTime)));
 }
 
 public VersionedValue moving(Token token)

git commit: fix changes

Updated Branches:
  refs/heads/trunk 1dacd2b6d - e5e72258e


fix changes


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e5e72258
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e5e72258
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e5e72258

Branch: refs/heads/trunk
Commit: e5e72258e3b000592861f8fb4bf3db863faa3913
Parents: 1dacd2b
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 15:58:57 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 15:58:57 2013 -0500

--
 CHANGES.txt | 2 --
 1 file changed, 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e5e72258/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1c5abd0..31cbb05 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -69,8 +69,6 @@
 
 
 1.2.7
- * Fix ReadResponseSerializer.serializedSize() for digest reads 
(CASSANDRA-5476)
- * allow sstable2json on 2i CFs (CASSANDRA-5694)
  * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
 
 1.2.6

[jira] [Comment Edited] (CASSANDRA-5704) Truncate flushes to disk again in 1.2, even with durable_writes=false


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694036#comment-13694036
 ] 

Christian Spriegel edited comment on CASSANDRA-5704 at 6/26/13 9:09 PM:


{quote}Data that was supposed to be gone, could reappear which feels 
semantically different to me.{quote}
I can understand that. My thought was that durable_writes already applies to 
deletes. So I thought why not also apply that pattern to truncate?


I could live with a new test.mode property though (in cassandra.yaml, 
right?). I would even go as far to say that this one should also deactivate all 
fsyncs in Cassandra :-)

Nevertheless, for practical reasons I like the idea of using the durable_writes 
flag, because it allows me to use have different behaviour on my dev-laptop 
for...
- ... one keyspace for my junits. These require constant truncation.
- ... my local installation of our application. Here I like a bit more data 
safety, because I not want to set up my testdata all the time.


  was (Author: christianmovi):
{quote}Data that was supposed to be gone, could reappear which feels 
semantically different to me.{quote}
I can understand that. My thought was that durable_writes already applies to 
deletes. So I thought why not also apply that pattern to truncate?


I could live with a new test.mode property though (in cassandra.yaml, 
right?). I would even go as far to say that this one should deactivate all 
fsyncs in Cassandra :-)

Nevertheless, for practical reasons I like the idea of using the durable_writes 
flag, because it allows me to use have different behaviour on my dev-laptop 
for...
- ... one keyspace for my junits. These require constant truncation.
- ... my local installation of our application. Here I like a bit more data 
safety, because I not want to set up my testdata all the time.

  
 Truncate flushes to disk again in 1.2, even with durable_writes=false
 -

 Key: CASSANDRA-5704
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5704
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.5
Reporter: Christian Spriegel
Assignee: Christian Spriegel
Priority: Minor
 Fix For: 1.2.7

 Attachments: CASSANDRA_5704_V1_1_2.patch, 
 CASSANDRA_5704_V1_trunk.patch


 I just upgraded my dev-environment to C* 1.2. Unfortunetaly 1.2 makes my 
 JUnit tests slow again, due to a blocking-flush in saveTruncationRecord().
 With Cassandra 1.1 truncate was very fast due to: CASSANDRA-4153
 My proposal is to make saveTruncationRecord() only flush when durableWrites 
 are enabled.
 My assumption is that if somebody turn off durable writes then he does not 
 mind if truncate is not guaranteed to be durable either.
 I successfully tested the following patch with my testsuite. Its as fast as 
 it was with 1.1 (maybe even faster!):
 {code}
 @@ -186,5 +186,8 @@ public class SystemTable
  String req = UPDATE system.%s SET truncated_at = truncated_at + %s 
 WHERE key = '%s';
  processInternal(String.format(req, LOCAL_CF, 
 truncationAsMapEntry(cfs, truncatedAt, position), LOCAL_KEY));
 -forceBlockingFlush(LOCAL_CF);
 +
 +KSMetaData ksm = Schema.instance.getKSMetaData(cfs.table.name);
 +if (ksm.durableWrites) // flush only when durable_writes are enabled
 +forceBlockingFlush(LOCAL_CF);
  }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5709) Convert pig example data population to cqlsh

Brandon Williams created CASSANDRA-5709:
---

 Summary: Convert pig example data population to cqlsh
 Key: CASSANDRA-5709
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5709
 Project: Cassandra
  Issue Type: Improvement
Affects Versions: 1.2.0
Reporter: Brandon Williams
Priority: Minor
 Fix For: 1.2.7


Currently, the pig examples populate cassandra via the cli.  However, post-5234 
we can use cql3 CFs and should provide examples of doing so without having two 
separate load files since cql3 can do everything the cli currently is.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694266#comment-13694266
 ] 

Deepak Rosario Pancras commented on CASSANDRA-5234:
---

@Alex: I did check out cassandra-1.2 and tried to apply the patch using patch 
-p1  5234-3-1.2branch.txt But it didnt work it threw and exception  is 
reserveed for future use.
So, I downloaded the patch 5234-3-1.2branch.txt and did a git apply and then 
committed the unstaged changes. Now when I browsed the file location cassandra 
/ src / java / org / apache / cassandra / hadoop / pig / the patch is not 
applied. Sorry for my naive questions, I am very new to git and cassandra.

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10

[
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694269#comment-13694269
]

Deepak Rosario Pancras commented on CASSANDRA-5234:
---

The patch is applied. But do I have to worry about the

5234-3-1.2branch.txt:22: trailing whitespace.
partitioner =
FBUtilities.newPartitioner(client.describe_partitioner());
5234-3-1.2branch.txt:735: trailing whitespace.

5234-3-1.2branch.txt:925: trailing whitespace.

5234-3-1.2branch.txt:1297: trailing whitespace.
}
5234-3-1.2branch.txt:1321: trailing whitespace.

Table created through CQL3 are not accessble to Pig 0.10

Key: CASSANDRA-5234
URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.2.1
Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
Fix For: 1.2.7

Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt,
5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx

Hi,
i have faced a bug when creating table through CQL3 and trying to load data
through pig 0.10 as follows:
java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
at
org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
at
org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
This effects from Simple table to table with compound key.

[1/2] Pig: support for cq3 tables Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-5234

Updated Branches:
  refs/heads/cassandra-1.2 54266ea70 - 33a3d2ca5


http://git-wip-us.apache.org/repos/asf/cassandra/blob/33a3d2ca/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java 
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
new file mode 100644
index 000..004b319
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -0,0 +1,447 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.pig;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+
+import org.apache.cassandra.db.IColumn;
+import org.apache.cassandra.db.Column;
+import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.hadoop.*;
+import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.hadoop.mapreduce.*;
+import org.apache.pig.Expression;
+import org.apache.pig.ResourceSchema;
+import org.apache.pig.ResourceSchema.ResourceFieldSchema;
+import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit;
+import org.apache.pig.data.*;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A LoadStoreFunc for retrieving data from and storing data to Cassandra
+ *
+ * A row from a standard CF will be returned as nested tuples: 
+ * (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).
+ */
+public class CqlStorage extends AbstractCassandraStorage
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlStorage.class);
+
+private RecordReaderMapString, ByteBuffer, MapString, ByteBuffer 
reader;
+private RecordWriterMapString, ByteBuffer, ListByteBuffer writer;
+
+private int pageSize = 1000;
+private String columns;
+private String outputQuery;
+private String whereClause;
+
+public CqlStorage()
+{
+this(1000);
+}
+
+/** @param limit number of CQL rows to fetch in a thrift request */
+public CqlStorage(int pageSize)
+{
+super();
+this.pageSize = pageSize;
+DEFAULT_INPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_OUTPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
+}   
+
+public void prepareToRead(RecordReader reader, PigSplit split)
+{
+this.reader = reader;
+}
+
+/** get next row */
+public Tuple getNext() throws IOException
+{
+try
+{
+// load the next pair
+if (!reader.nextKeyValue())
+return null;
+
+CfDef cfDef = getCfDef(loadSignature);
+MapString, ByteBuffer keys = reader.getCurrentKey();
+MapString, ByteBuffer columns = reader.getCurrentValue();
+assert keys != null  columns != null;
+
+// add key columns to the map
+for (Map.EntryString,ByteBuffer key : keys.entrySet())
+columns.put(key.getKey(), key.getValue());
+
+Tuple tuple = 
TupleFactory.getInstance().newTuple(cfDef.column_metadata.size());
+IteratorColumnDef itera = cfDef.column_metadata.iterator();
+int i = 0;
+while (itera.hasNext())
+{
+ColumnDef cdef = itera.next();
+ByteBuffer columnValue = 
columns.get(ByteBufferUtil.string(cdef.name.duplicate()));
+if (columnValue != null)
+{
+IColumn column = new Column(cdef.name, columnValue);
+tuple.set(i, columnToTuple(column, cfDef, 
UTF8Type.instance));
+}
+else
+tuple.set(i, TupleFactory.getInstance().newTuple());
+i++;
+}
+return tuple;
+}
+catch (InterruptedException e)
+{
+throw new IOException(e.getMessage());
+}
+}
+
+/** set read configuration

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694284#comment-13694284
 ] 

Brandon Williams commented on CASSANDRA-5234:
-

Committed, and created CASSANDRA-5709 for example follow-up.  Thanks!

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694286#comment-13694286
 ] 

Brandon Williams commented on CASSANDRA-5234:
-

[~tharigopla] it's in the 1.2 branch now, just do a git pull.

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[3/3] git commit: Ghetto merge branch 'cassandra-1.2' into trunk pending another patch for 5234

Ghetto merge branch 'cassandra-1.2' into trunk pending another patch for 5234


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d4e470b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d4e470b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d4e470b

Branch: refs/heads/trunk
Commit: 3d4e470baa0cf53ef84a36f96db6645e17770fbf
Parents: e5e7225 33a3d2c
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 16:58:46 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 16:58:46 2013 -0500

--

--

[1/3] Pig: support for cq3 tables Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-5234

Updated Branches:
  refs/heads/trunk e5e72258e - 3d4e470ba


http://git-wip-us.apache.org/repos/asf/cassandra/blob/33a3d2ca/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java 
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
new file mode 100644
index 000..004b319
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -0,0 +1,447 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.pig;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+
+import org.apache.cassandra.db.IColumn;
+import org.apache.cassandra.db.Column;
+import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.hadoop.*;
+import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.hadoop.mapreduce.*;
+import org.apache.pig.Expression;
+import org.apache.pig.ResourceSchema;
+import org.apache.pig.ResourceSchema.ResourceFieldSchema;
+import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit;
+import org.apache.pig.data.*;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A LoadStoreFunc for retrieving data from and storing data to Cassandra
+ *
+ * A row from a standard CF will be returned as nested tuples: 
+ * (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).
+ */
+public class CqlStorage extends AbstractCassandraStorage
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlStorage.class);
+
+private RecordReaderMapString, ByteBuffer, MapString, ByteBuffer 
reader;
+private RecordWriterMapString, ByteBuffer, ListByteBuffer writer;
+
+private int pageSize = 1000;
+private String columns;
+private String outputQuery;
+private String whereClause;
+
+public CqlStorage()
+{
+this(1000);
+}
+
+/** @param limit number of CQL rows to fetch in a thrift request */
+public CqlStorage(int pageSize)
+{
+super();
+this.pageSize = pageSize;
+DEFAULT_INPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_OUTPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
+}   
+
+public void prepareToRead(RecordReader reader, PigSplit split)
+{
+this.reader = reader;
+}
+
+/** get next row */
+public Tuple getNext() throws IOException
+{
+try
+{
+// load the next pair
+if (!reader.nextKeyValue())
+return null;
+
+CfDef cfDef = getCfDef(loadSignature);
+MapString, ByteBuffer keys = reader.getCurrentKey();
+MapString, ByteBuffer columns = reader.getCurrentValue();
+assert keys != null  columns != null;
+
+// add key columns to the map
+for (Map.EntryString,ByteBuffer key : keys.entrySet())
+columns.put(key.getKey(), key.getValue());
+
+Tuple tuple = 
TupleFactory.getInstance().newTuple(cfDef.column_metadata.size());
+IteratorColumnDef itera = cfDef.column_metadata.iterator();
+int i = 0;
+while (itera.hasNext())
+{
+ColumnDef cdef = itera.next();
+ByteBuffer columnValue = 
columns.get(ByteBufferUtil.string(cdef.name.duplicate()));
+if (columnValue != null)
+{
+IColumn column = new Column(cdef.name, columnValue);
+tuple.set(i, columnToTuple(column, cfDef, 
UTF8Type.instance));
+}
+else
+tuple.set(i, TupleFactory.getInstance().newTuple());
+i++;
+}
+return tuple;
+}
+catch (InterruptedException e)
+{
+throw new IOException(e.getMessage());
+}
+}
+
+/** set read configuration settings */
+

git commit: cqlsh: Don't show 'null' in place of empty values

Updated Branches:
  refs/heads/cassandra-1.2 33a3d2ca5 - e4050e609


cqlsh: Don't show 'null' in place of empty values

patch by Aleksey Yeschenko; reviewed by Brandon Williams for
CASSANDRA-5675


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4050e60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4050e60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4050e60

Branch: refs/heads/cassandra-1.2
Commit: e4050e609cc4bdc4b60ffb829774a7ce09dad726
Parents: 33a3d2c
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 00:36:46 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 01:03:07 2013 +0300

--
 CHANGES.txt |  1 +
 bin/cqlsh   | 30 ++--
 pylib/cqlshlib/displaying.py|  4 ++--
 pylib/cqlshlib/formatting.py|  6 +++--
 pylib/cqlshlib/test/test_cqlsh_output.py| 10 
 pylib/cqlshlib/test/test_keyspace_init2.cql |  4 
 6 files changed, 39 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4050e60/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 24d4c9e..e7e8652 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 1.2.7
  * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
  * Pig: support for cql3 tables (CASSANDRA-5234)
+ * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 
 1.2.6
  * Fix tracing when operation completes before all responses arrive 
(CASSANDRA-5668)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4050e60/bin/cqlsh
--
diff --git a/bin/cqlsh b/bin/cqlsh
index 70b70f5..9f1e6cf 100755
--- a/bin/cqlsh
+++ b/bin/cqlsh
@@ -567,6 +567,17 @@ class Shell(cmd.Cmd):
 def myformat_colname(self, name, nametype):
 return self.myformat_value(name, nametype, colormap=COLUMN_NAME_COLORS)
 
+# cql/cursor.py:Cursor.decode_row() function, modified to not turn '' into 
None.
+def decode_row(self, cursor, row):
+values = []
+bytevals = cursor.columnvalues(row)
+for val, vtype, nameinfo in zip(bytevals, cursor.column_types, 
cursor.name_info):
+if val == '':
+values.append(val)
+else:
+values.append(cursor.decoder.decode_value(val, vtype, 
nameinfo[0]))
+return values
+
 def report_connection(self):
 self.show_host()
 self.show_version()
@@ -1115,7 +1126,7 @@ class Shell(cmd.Cmd):
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) in 
enumerate(colnames)]
 formatted_names = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-formatted_values = [map(self.myformat_value, row, cursor.column_types) 
for row in cursor]
+formatted_values = [map(self.myformat_value, self.decode_row(cursor, 
row), cursor.column_types) for row in cursor.result]
 if self.expand_enabled:
 self.print_formatted_result_vertically(formatted_names, 
formatted_values)
 else:
@@ -1153,11 +1164,12 @@ class Shell(cmd.Cmd):
 self.writeresult('')
 
 def print_dynamic_result(self, cursor):
-for row in cursor:
+for row in cursor.result:
+cursor.fetchone()
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) 
in enumerate(colnames)]
 colnames = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-colvals = map(self.myformat_value, row, cursor.column_types)
+colvals = map(self.myformat_value, self.decode_row(cursor, row), 
cursor.column_types)
 line = ' | '.join('%s,%s' % (n.coloredval, v.coloredval) for (n, 
v) in zip(colnames, colvals))
 self.writeresult(' ' + line)
 
@@ -1680,14 +1692,18 @@ class Shell(cmd.Cmd):
 def do_import_row(self, columns, nullval, layout, row):
 rowmap = {}
 for name, value in zip(columns, row):
+type = layout.get_column(name).cqltype
+if issubclass(type, ReversedType):
+type = type.subtypes[0]
+cqltype = type.cql_parameterized_type()
+
 if value != nullval:
-type = layout.get_column(name).cqltype
-if issubclass(type, ReversedType):
-type = type.subtypes[0]
-if type.cql_parameterized_type() in ('ascii', 'text', 
'timestamp', 'inet'):
+if cqltype in

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data

[
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694292#comment-13694292
]

Pavel Yaskevich commented on CASSANDRA-5661:

bq. The problem is that most of those sstables have 1 MB or more of CRAR
buffers sitting there waiting to be used. If we do a timer with an absolute
cap, even better, but we need some kind of cap.

1MB actually means that it's around 12-14 caches reads which is not even half
of default concurrency limit, we can start with expiring for 1.2.7 as it
doesn't require adding options (and maybe disable caching with LCS), then for
2.0 we can add cap limit per CF or even global into yaml.

bq. The compression overhead itself isn't really an issue, the issue is the
cached CRAR objects/buffers.

It is for such a small files because it's not just buffers but also compression
metadata per SSTable although you can't see it in histogram/tree as it was
moved off heap, I guess it's an artifact of LCS that we can't do anything about.

This all got me wondering what is BF space overhead LCS vs. STCS, can you
please check?

Discard pooled readers for cold data

Attachments: DominatorTree.png, Histogram.png

[jira] [Updated] (CASSANDRA-5706) OOM while loading key cache at startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5706:
--

Affects Version/s: (was: 1.2.5)
   1.2.0

(actually, annotate says this dates back to CASSANDRA-3762)

 OOM while loading key cache at startup
 --

 Key: CASSANDRA-5706
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5706
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Fabien Rousseau
Assignee: Fabien Rousseau
 Fix For: 1.2.7

 Attachments: 5706-OOM-while-loading-key-cache-at-startup.patch


 Steps to be able to reproduce it :
  - have a heap of 1Gb
  - have a saved key cache without the SSTables
 When looking at KeyCacheSerializer.serialize : it always writes a Boolean
 When looking at KeyCacheSerializer.deserialize : no Boolean is read if 
 SSTable is missing...
 In case of a promoted index, RowIndexEntry.serializer.skip(...) should be 
 called rather than RowIndexEntry.serializer.skipPromotedIndex(...) (again for 
 symmetry between serialization/deserialization)
 Attached is a proposed patch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5706) OOM while loading key cache at startup


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5706:
--

Attachment: 5706-v2-txt

Your analysis is correct; attached is slightly revised patch to emphasize that 
we already read exactly one boolean

 OOM while loading key cache at startup
 --

 Key: CASSANDRA-5706
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5706
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Fabien Rousseau
Assignee: Fabien Rousseau
 Fix For: 1.2.7

 Attachments: 5706-OOM-while-loading-key-cache-at-startup.patch, 
 5706-v2-txt


 Steps to be able to reproduce it :
  - have a heap of 1Gb
  - have a saved key cache without the SSTables
 When looking at KeyCacheSerializer.serialize : it always writes a Boolean
 When looking at KeyCacheSerializer.deserialize : no Boolean is read if 
 SSTable is missing...
 In case of a promoted index, RowIndexEntry.serializer.skip(...) should be 
 called rather than RowIndexEntry.serializer.skipPromotedIndex(...) (again for 
 symmetry between serialization/deserialization)
 Attached is a proposed patch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694298#comment-13694298
 ] 

Deepak Rosario Pancras commented on CASSANDRA-5234:
---

Thanks Brandon

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694299#comment-13694299
 ] 

Jonathan Ellis commented on CASSANDRA-5661:
---

bq. it's not just buffers but also compression metadata per SSTable although 
you can't see it in histogram/tree as it was moved off heap

The on-heap part is bad enough :)

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7

 Attachments: DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/5] git commit: cqlsh: Don't show 'null' in place of empty values

Updated Branches:
  refs/heads/trunk 3d4e470ba - 6def8223f


cqlsh: Don't show 'null' in place of empty values

patch by Aleksey Yeschenko; reviewed by Brandon Williams for
CASSANDRA-5675


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/340a6689
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/340a6689
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/340a6689

Branch: refs/heads/trunk
Commit: 340a66891c7eb5e550b8a72401de930cb4c5c392
Parents: 54266ea
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 00:36:46 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 00:36:46 2013 +0300

--
 CHANGES.txt |  1 +
 bin/cqlsh   | 30 ++--
 pylib/cqlshlib/displaying.py|  4 ++--
 pylib/cqlshlib/formatting.py|  6 +++--
 pylib/cqlshlib/test/test_cqlsh_output.py| 10 
 pylib/cqlshlib/test/test_keyspace_init2.cql |  4 
 6 files changed, 39 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/340a6689/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cb3fede..94830fd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
 1.2.7
  * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
+ * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 
 1.2.6
  * Fix tracing when operation completes before all responses arrive 
(CASSANDRA-5668)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/340a6689/bin/cqlsh
--
diff --git a/bin/cqlsh b/bin/cqlsh
index 70b70f5..9f1e6cf 100755
--- a/bin/cqlsh
+++ b/bin/cqlsh
@@ -567,6 +567,17 @@ class Shell(cmd.Cmd):
 def myformat_colname(self, name, nametype):
 return self.myformat_value(name, nametype, colormap=COLUMN_NAME_COLORS)
 
+# cql/cursor.py:Cursor.decode_row() function, modified to not turn '' into 
None.
+def decode_row(self, cursor, row):
+values = []
+bytevals = cursor.columnvalues(row)
+for val, vtype, nameinfo in zip(bytevals, cursor.column_types, 
cursor.name_info):
+if val == '':
+values.append(val)
+else:
+values.append(cursor.decoder.decode_value(val, vtype, 
nameinfo[0]))
+return values
+
 def report_connection(self):
 self.show_host()
 self.show_version()
@@ -1115,7 +1126,7 @@ class Shell(cmd.Cmd):
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) in 
enumerate(colnames)]
 formatted_names = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-formatted_values = [map(self.myformat_value, row, cursor.column_types) 
for row in cursor]
+formatted_values = [map(self.myformat_value, self.decode_row(cursor, 
row), cursor.column_types) for row in cursor.result]
 if self.expand_enabled:
 self.print_formatted_result_vertically(formatted_names, 
formatted_values)
 else:
@@ -1153,11 +1164,12 @@ class Shell(cmd.Cmd):
 self.writeresult('')
 
 def print_dynamic_result(self, cursor):
-for row in cursor:
+for row in cursor.result:
+cursor.fetchone()
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) 
in enumerate(colnames)]
 colnames = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-colvals = map(self.myformat_value, row, cursor.column_types)
+colvals = map(self.myformat_value, self.decode_row(cursor, row), 
cursor.column_types)
 line = ' | '.join('%s,%s' % (n.coloredval, v.coloredval) for (n, 
v) in zip(colnames, colvals))
 self.writeresult(' ' + line)
 
@@ -1680,14 +1692,18 @@ class Shell(cmd.Cmd):
 def do_import_row(self, columns, nullval, layout, row):
 rowmap = {}
 for name, value in zip(columns, row):
+type = layout.get_column(name).cqltype
+if issubclass(type, ReversedType):
+type = type.subtypes[0]
+cqltype = type.cql_parameterized_type()
+
 if value != nullval:
-type = layout.get_column(name).cqltype
-if issubclass(type, ReversedType):
-type = type.subtypes[0]
-if type.cql_parameterized_type() in ('ascii', 'text', 
'timestamp', 'inet'):
+if cqltype in ('ascii', 'text', 'timestamp', 'inet'):

[3/5] git commit: cqlsh: Don't show 'null' in place of empty values

cqlsh: Don't show 'null' in place of empty values

patch by Aleksey Yeschenko; reviewed by Brandon Williams for
CASSANDRA-5675


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4050e60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4050e60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4050e60

Branch: refs/heads/trunk
Commit: e4050e609cc4bdc4b60ffb829774a7ce09dad726
Parents: 33a3d2c
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 00:36:46 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 01:03:07 2013 +0300

--
 CHANGES.txt |  1 +
 bin/cqlsh   | 30 ++--
 pylib/cqlshlib/displaying.py|  4 ++--
 pylib/cqlshlib/formatting.py|  6 +++--
 pylib/cqlshlib/test/test_cqlsh_output.py| 10 
 pylib/cqlshlib/test/test_keyspace_init2.cql |  4 
 6 files changed, 39 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4050e60/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 24d4c9e..e7e8652 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
 1.2.7
  * Fix serialization of the LEFT gossip value (CASSANDRA-5696)
  * Pig: support for cql3 tables (CASSANDRA-5234)
+ * cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
 
 1.2.6
  * Fix tracing when operation completes before all responses arrive 
(CASSANDRA-5668)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4050e60/bin/cqlsh
--
diff --git a/bin/cqlsh b/bin/cqlsh
index 70b70f5..9f1e6cf 100755
--- a/bin/cqlsh
+++ b/bin/cqlsh
@@ -567,6 +567,17 @@ class Shell(cmd.Cmd):
 def myformat_colname(self, name, nametype):
 return self.myformat_value(name, nametype, colormap=COLUMN_NAME_COLORS)
 
+# cql/cursor.py:Cursor.decode_row() function, modified to not turn '' into 
None.
+def decode_row(self, cursor, row):
+values = []
+bytevals = cursor.columnvalues(row)
+for val, vtype, nameinfo in zip(bytevals, cursor.column_types, 
cursor.name_info):
+if val == '':
+values.append(val)
+else:
+values.append(cursor.decoder.decode_value(val, vtype, 
nameinfo[0]))
+return values
+
 def report_connection(self):
 self.show_host()
 self.show_version()
@@ -1115,7 +1126,7 @@ class Shell(cmd.Cmd):
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) in 
enumerate(colnames)]
 formatted_names = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-formatted_values = [map(self.myformat_value, row, cursor.column_types) 
for row in cursor]
+formatted_values = [map(self.myformat_value, self.decode_row(cursor, 
row), cursor.column_types) for row in cursor.result]
 if self.expand_enabled:
 self.print_formatted_result_vertically(formatted_names, 
formatted_values)
 else:
@@ -1153,11 +1164,12 @@ class Shell(cmd.Cmd):
 self.writeresult('')
 
 def print_dynamic_result(self, cursor):
-for row in cursor:
+for row in cursor.result:
+cursor.fetchone()
 colnames = [d[0] for d in cursor.description]
 colnames_t = [(name, self.get_nametype(cursor, n)) for (n, name) 
in enumerate(colnames)]
 colnames = [self.myformat_colname(name, nametype) for (name, 
nametype) in colnames_t]
-colvals = map(self.myformat_value, row, cursor.column_types)
+colvals = map(self.myformat_value, self.decode_row(cursor, row), 
cursor.column_types)
 line = ' | '.join('%s,%s' % (n.coloredval, v.coloredval) for (n, 
v) in zip(colnames, colvals))
 self.writeresult(' ' + line)
 
@@ -1680,14 +1692,18 @@ class Shell(cmd.Cmd):
 def do_import_row(self, columns, nullval, layout, row):
 rowmap = {}
 for name, value in zip(columns, row):
+type = layout.get_column(name).cqltype
+if issubclass(type, ReversedType):
+type = type.subtypes[0]
+cqltype = type.cql_parameterized_type()
+
 if value != nullval:
-type = layout.get_column(name).cqltype
-if issubclass(type, ReversedType):
-type = type.subtypes[0]
-if type.cql_parameterized_type() in ('ascii', 'text', 
'timestamp', 'inet'):
+if cqltype in ('ascii', 'text', 'timestamp', 'inet'):
 rowmap[name] =

[5/5] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6def8223
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6def8223
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6def8223

Branch: refs/heads/trunk
Commit: 6def8223f5e0dae5fae051986f5f428270db580f
Parents: d8302d7 e4050e6
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 01:15:04 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 01:15:04 2013 +0300

--

--

[4/5] git commit: Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/cassandra into trunk

Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/cassandra into 
trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d8302d79
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d8302d79
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d8302d79

Branch: refs/heads/trunk
Commit: d8302d79a85da676e483274599bcac6b15e3
Parents: f897843 3d4e470
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 01:12:50 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 01:12:50 2013 +0300

--

--

[2/5] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk

Conflicts:
bin/cqlsh
pylib/cqlshlib/test/test_cqlsh_output.py
pylib/cqlshlib/test/test_keyspace_init2.cql


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f8978433
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f8978433
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f8978433

Branch: refs/heads/trunk
Commit: f89784339c87d17152d80ed503703ba5b58e45be
Parents: e5e7225 340a668
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Jun 27 01:01:18 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Jun 27 01:01:18 2013 +0300

--
 CHANGES.txt  |  1 +
 bin/cqlsh| 25 ++-
 pylib/cqlshlib/displaying.py |  4 ++--
 pylib/cqlshlib/formatting.py |  6 --
 pylib/cqlshlib/test/test_cqlsh_completion.py |  2 +-
 pylib/cqlshlib/test/test_cqlsh_output.py | 15 +-
 pylib/cqlshlib/test/test_keyspace_init.cql   |  6 ++
 7 files changed, 44 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f8978433/CHANGES.txt
--
diff --cc CHANGES.txt
index 31cbb05,94830fd..59c9dab
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,75 -1,6 +1,76 @@@
 +2.0
 + * Removed on-heap row cache (CASSANDRA-5348)
 + * use nanotime consistently for node-local timeouts (CASSANDRA-5581)
 + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577)
 + * Experimental triggers (CASSANDRA-1311)
 + * JEMalloc support for off-heap allocation (CASSANDRA-3997)
 + * Single-pass compaction (CASSANDRA-4180)
 + * Removed token range bisection (CASSANDRA-5518)
 + * Removed compatibility with pre-1.2.5 sstables and network messages
 +   (CASSANDRA-5511)
 + * removed PBSPredictor (CASSANDRA-5455)
 + * CAS support (CASSANDRA-5062, 5441, 5442, 5443, 5619)
 + * Leveled compaction performs size-tiered compactions in L0 
 +   (CASSANDRA-5371, 5439)
 + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339)
 + * Log when a node is down longer than the hint window (CASSANDRA-4554)
 + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917)
 + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407)
 + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430)
 + * Change Message IDs to ints (CASSANDRA-5307)
 + * Move sstable level information into the Stats component, removing the
 +   need for a separate Manifest file (CASSANDRA-4872)
 + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199)
 + * make index_interval configurable per columnfamily (CASSANDRA-3961, 
CASSANDRA-5650)
 + * add default_time_to_live (CASSANDRA-3974)
 + * add memtable_flush_period_in_ms (CASSANDRA-4237)
 + * replace supercolumns internally by composites (CASSANDRA-3237, 5123)
 + * upgrade thrift to 0.9.0 (CASSANDRA-3719)
 + * drop unnecessary keyspace parameter from user-defined compaction API 
 +   (CASSANDRA-5139)
 + * more robust solution to incomplete compactions + counters (CASSANDRA-5151)
 + * Change order of directory searching for c*.in.sh (CASSANDRA-3983)
 + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271)
 + * Allow custom configuration loader (CASSANDRA-5045)
 + * Remove memory emergency pressure valve logic (CASSANDRA-3534)
 + * Reduce request latency with eager retry (CASSANDRA-4705)
 + * cqlsh: Remove ASSUME command (CASSANDRA-5331)
 + * Rebuild BF when loading sstables if bloom_filter_fp_chance
 +   has changed since compaction (CASSANDRA-5015)
 + * remove row-level bloom filters (CASSANDRA-4885)
 + * Change Kernel Page Cache skipping into row preheating (disabled by default)
 +   (CASSANDRA-4937)
 + * Improve repair by deciding on a gcBefore before sending
 +   out TreeRequests (CASSANDRA-4932)
 + * Add an official way to disable compactions (CASSANDRA-5074)
 + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919)
 + * Add binary protocol versioning (CASSANDRA-5436)
 + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530)
 + * Add alias support to SELECT statement (CASSANDRA-5075)
 + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541)
 + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579)
 + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585)
 + * Track max/min column names in sstables to be able to optimize slice
 +   queries (CASSANDRA-5514, CASSANDRA-5595, CASSANDRA-5600)
 + * Binary protocol: allow batching already prepared statements 
(CASSANDRA-4693)
 + * Allow preparing timestamp, ttl and limit in CQL3 queries (CASSANDRA-4450)
 + * Support native link w/o

[jira] [Updated] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-5234:


Attachment: 5234-3-trunk.txt

5234-3-trunk.txt patch for trunk branch is attached.

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234-3-trunk.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-5234:


Attachment: (was: 5234-3-trunk.txt)

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234-3-trunk.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5234) Table created through CQL3 are not accessble to Pig 0.10


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-5234:


Attachment: 5234-3-trunk.txt

 Table created through CQL3 are not accessble to Pig 0.10
 

 Key: CASSANDRA-5234
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5234
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.7

 Attachments: 5234-1-1.2-patch.txt, 5234-1.2-patch.txt, 
 5234-2-1.2branch.txt, 5234-3-1.2branch.txt, 5234-3-trunk.txt, 5234.tx


 Hi,
   i have faced a bug when creating table through CQL3 and trying to load data 
 through pig 0.10 as follows:
 java.lang.RuntimeException: Column family 'abc' not found in keyspace 'XYZ'
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.initSchema(CassandraStorage.java:1112)
   at 
 org.apache.cassandra.hadoop.pig.CassandraStorage.setLocation(CassandraStorage.java:615).
 This effects from Simple table to table with compound key. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/2] Pig: support for cql3 tables Patch by Alex Liu, reviewed by brandonwilliams for CASSANDRA-5234

Updated Branches:
  refs/heads/trunk 6def8223f - 764bcd3f3


http://git-wip-us.apache.org/repos/asf/cassandra/blob/764bcd3f/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java 
b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
new file mode 100644
index 000..b0d9dee
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java
@@ -0,0 +1,446 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.pig;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+
+import org.apache.cassandra.db.Column;
+import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.hadoop.*;
+import org.apache.cassandra.hadoop.cql3.CqlConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.hadoop.mapreduce.*;
+import org.apache.pig.Expression;
+import org.apache.pig.ResourceSchema;
+import org.apache.pig.ResourceSchema.ResourceFieldSchema;
+import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit;
+import org.apache.pig.data.*;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A LoadStoreFunc for retrieving data from and storing data to Cassandra
+ *
+ * A row from a standard CF will be returned as nested tuples: 
+ * (((key1, value1), (key2, value2)), ((name1, val1), (name2, val2))).
+ */
+public class CqlStorage extends AbstractCassandraStorage
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlStorage.class);
+
+private RecordReaderMapString, ByteBuffer, MapString, ByteBuffer 
reader;
+private RecordWriterMapString, ByteBuffer, ListByteBuffer writer;
+
+private int pageSize = 1000;
+private String columns;
+private String outputQuery;
+private String whereClause;
+
+public CqlStorage()
+{
+this(1000);
+}
+
+/** @param limit number of CQL rows to fetch in a thrift request */
+public CqlStorage(int pageSize)
+{
+super();
+this.pageSize = pageSize;
+DEFAULT_INPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlPagingInputFormat;
+DEFAULT_OUTPUT_FORMAT = 
org.apache.cassandra.hadoop.cql3.CqlOutputFormat;
+}   
+
+public void prepareToRead(RecordReader reader, PigSplit split)
+{
+this.reader = reader;
+}
+
+/** get next row */
+public Tuple getNext() throws IOException
+{
+try
+{
+// load the next pair
+if (!reader.nextKeyValue())
+return null;
+
+CfDef cfDef = getCfDef(loadSignature);
+MapString, ByteBuffer keys = reader.getCurrentKey();
+MapString, ByteBuffer columns = reader.getCurrentValue();
+assert keys != null  columns != null;
+
+// add key columns to the map
+for (Map.EntryString,ByteBuffer key : keys.entrySet())
+columns.put(key.getKey(), key.getValue());
+
+Tuple tuple = 
TupleFactory.getInstance().newTuple(cfDef.column_metadata.size());
+IteratorColumnDef itera = cfDef.column_metadata.iterator();
+int i = 0;
+while (itera.hasNext())
+{
+ColumnDef cdef = itera.next();
+ByteBuffer columnValue = 
columns.get(ByteBufferUtil.string(cdef.name.duplicate()));
+if (columnValue != null)
+{
+Column column = new Column(cdef.name, columnValue);
+tuple.set(i, columnToTuple(column, cfDef, 
UTF8Type.instance));
+}
+else
+tuple.set(i, TupleFactory.getInstance().newTuple());
+i++;
+}
+return tuple;
+}
+catch (InterruptedException e)
+{
+throw new IOException(e.getMessage());
+}
+}
+
+/** set read configuration settings */
+public void setLocation(String

[jira] [Updated] (CASSANDRA-5429) Update scrub and scrubtest for single-pass compaction format


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-5429:
---

Attachment: 5429-v1.diff

Attached patch fixes Scrubber to read/scrub Descriptor versions ic and ja. 

I tested by hand using tables created from both 1.2.6 and 2.0, and was able to 
successfully scrub both. It'll take a day or two to sort out the test (revive 
ScrubTest and get some relevant test sstables in place), but the core work that 
we'll need in 2.0 is in this patch.

 Update scrub and scrubtest for single-pass compaction format
 

 Key: CASSANDRA-5429
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5429
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 2.0 beta 1
Reporter: Jonathan Ellis
Assignee: Jason Brown
 Fix For: 2.0 beta 1

 Attachments: 5429-v1.diff




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694344#comment-13694344
 ] 

Pavel Yaskevich commented on CASSANDRA-5661:


After thinking about this more as LCS produces similar (same) sized files there 
is actually no benefit of caching [C]RAR instances. I think we should do 
caching only with STCS + expiring as it introduces less memory overhead per 
file for [C]RAR.

I also want to share my worries about LCS using 5MB file size with mmap and no 
compression: default vm.max_map_count is 65536 (maximum number of memory 
mappings per process) which is around 322GB of data, we need to mention 
somewhere that if somebody wants to run with mmap + LCS they need to adjust 
that setting. Another thing that worries me is that performance of mmap, munmap 
and similar + minor/major page faults would degrade logarithmically as memory 
mappings are handled by red-black tree and with constant compaction we would be 
burning a lot more cpu on tree balancing as dataset grows.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7

 Attachments: DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Add forgotten updates

Updated Branches:
  refs/heads/trunk 764bcd3f3 - 168482bc1


Add forgotten updates


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/168482bc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/168482bc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/168482bc

Branch: refs/heads/trunk
Commit: 168482bc113616206701d86fd9db29849a3635b5
Parents: 764bcd3
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 26 19:10:05 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 26 19:10:05 2013 -0500

--
 examples/pig/README.txt| 21 ++---
 examples/pig/test/test_storage.pig |  2 +-
 2 files changed, 19 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/168482bc/examples/pig/README.txt
--
diff --git a/examples/pig/README.txt b/examples/pig/README.txt
index e3d9af6..6dc0937 100644
--- a/examples/pig/README.txt
+++ b/examples/pig/README.txt
@@ -32,7 +32,10 @@ for input and output:
 * PIG_OUTPUT_RPC_PORT : the port thrift is listening on for writing
 * PIG_OUTPUT_PARTITIONER : cluster partitioner for writing
 
-Then you can run it like this:
+CassandraStorage
+
+
+The CassandraStorage class is for any non-CQL3 ColumnFamilies you may have.  
For CQL3 support, refer to the CqlStorage section.
 
 examples/pig$ bin/pig_cassandra -x local example-script.pig
 
@@ -71,8 +74,8 @@ already exist for this to work.
 
 See the example in test/ to see how schema is inferred.
 
-Advanced Options
-
+Advanced Options for CassandraStorage
+=
 
 The following environment variables default to false but can be set to true to 
enable them:
 
@@ -92,3 +95,15 @@ PIG_INPUT_SPLIT_SIZE: this sets the split size passed to 
Hadoop, controlling
   the amount of mapper tasks created.  This can also be 
set in the LOAD url by
   adding the 'split_size=X' parameter, where X is an 
integer amount for the size.
 
+CqlStorage
+==
+
+The CqlStorage class is somewhat similar to CassandraStorage, but it can work 
with CQL3-defined ColumnFamilies.  The main difference is in the URL format:
+
+cql://[username:password@]keyspace/columnfamily[?[page_size=size][columns=col1,col2][output_query=prepared_statement][where_clause=clause][split_size=size][partitioner=partitioner]]
+
+Which in grunt, the simplest example would look like:
+
+grunt rows = LOAD 'cql://MyKeyspace/MyColumnFamily' USING CqlStorage();
+
+CqlStorage handles wide rows automatically and thus has no separate flag for 
this.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/168482bc/examples/pig/test/test_storage.pig
--
diff --git a/examples/pig/test/test_storage.pig 
b/examples/pig/test/test_storage.pig
index 93dd91f..026cb02 100644
--- a/examples/pig/test/test_storage.pig
+++ b/examples/pig/test/test_storage.pig
@@ -1,4 +1,4 @@
-rows = LOAD 'cassandra://PigTest/SomeApp?widerows=true' USING 
CassandraStorage();
+rows = LOAD 'cassandra://PigTest/SomeApp' USING CassandraStorage();
 -- full copy
 STORE rows INTO 'cassandra://PigTest/CopyOfSomeApp' USING CassandraStorage();
 -- single tuple

[jira] [Commented] (CASSANDRA-5700) compact storage value restriction message confusing

2013-06-26 Thread Dave Brosius (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694383#comment-13694383
 ] 

Dave Brosius commented on CASSANDRA-5700:
-

much better... no need to mention clustering key? is that considerered part of 
the non-primary-key column ?

 compact storage value restriction message confusing
 ---

 Key: CASSANDRA-5700
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5700
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.2.5
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 1.2.7

 Attachments: 5700.txt


 i have a compact storage value column name (user) the same as another column 
 family name (user) and was getting the error
 Restricting the value of a compact CF (user) is not supported
 which was very confusing
 changed message to
 Restricting the value (user) of a compact CF is not supported
 (tackling the big problems)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694410#comment-13694410
 ] 

Jonathan Ellis commented on CASSANDRA-5661:
---

I don't see what similar-sized files has to do with it.  CRAR buffer size is 
independent of file size, and that's what causes the fragmentation.

 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7

 Attachments: DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5661) Discard pooled readers for cold data


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694410#comment-13694410
 ] 

Jonathan Ellis edited comment on CASSANDRA-5661 at 6/27/13 1:36 AM:


I don't see what similar-sized files has to do with it.  CRAR buffer size is 
independent of file size, and that's what causes the fragmentation.  (Zeroing 
out large buffers isn't free, either.)

  was (Author: jbellis):
I don't see what similar-sized files has to do with it.  CRAR buffer size 
is independent of file size, and that's what causes the fragmentation.
  
 Discard pooled readers for cold data
 

 Key: CASSANDRA-5661
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.1
Reporter: Jonathan Ellis
Assignee: Pavel Yaskevich
 Fix For: 1.2.7

 Attachments: DominatorTree.png, Histogram.png


 Reader pooling was introduced in CASSANDRA-4942 but pooled 
 RandomAccessReaders are never cleaned up until the SSTableReader is closed.  
 So memory use is the worst case simultaneous RAR we had open for this file, 
 forever.
 We should introduce a global limit on how much memory to use for RAR, and 
 evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5700) compact storage value restriction message confusing


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13694412#comment-13694412
 ] 

Jonathan Ellis commented on CASSANDRA-5700:
---

Cluster columns are part of the primary key.

 compact storage value restriction message confusing
 ---

 Key: CASSANDRA-5700
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5700
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 1.2.5
Reporter: Dave Brosius
Assignee: Dave Brosius
Priority: Trivial
 Fix For: 1.2.7

 Attachments: 5700.txt


 i have a compact storage value column name (user) the same as another column 
 family name (user) and was getting the error
 Restricting the value of a compact CF (user) is not supported
 which was very confusing
 changed message to
 Restricting the value (user) of a compact CF is not supported
 (tackling the big problems)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data